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Foreword 


Robotics  and  Computer  Assisted  Medical  Interventions  (RCAMI)  has  emerged  as  a  rapidly 
evolving  area  of  research  and  development  with  great  potential  for  improving  clinical  out¬ 
comes  while  also  reducing  patient  morbidity  and  the  cost  of  health  care.  The  first  generation 
of  computer-assisted  systems  are  being  utilized  in  operating  rooms  and  hospitals  throughout 
the  world.  Advancing  RCAMI  technology  to  the  next  levels  of  development  and  utilization 
requires  collaboration  among  the  disciplines  of  engineering,  science  and  medicine.  The 
RCAMI  workshop  brought  together  clinicians,  engineers,  scientists,  and  industrial  representa¬ 
tives.  Through  intensive  discussions,  the  participants  defined  current  status  and  clinical  appli¬ 
cations  of  these  technologies,  and  explored  future  directions  and  requirements. 

The  RCAMI  Workshop  was  truly  an  international  event  that  owes  its  success  to  the  contribu¬ 
tions  of  the  participants,  section  leaders,  and  sponsors.  We  would  like  to  thank  those  who  par¬ 
ticipated  in  the  workshop  for  their  enthusiasm,  dedication,  hard  work  and  their  contributions 
toward  the  development  of  this  report.  We  would  also  like  to  thank  all  of  our  sponsors,  and 
especially  the  National  Science  Foundation  and  Gil  Devey  for  providing  the  seed  support  and 
guidance  in  developing  the  concept  for  the  workshop.  Special  thanks  to  our  hosts  in  Bristol, 
England,  the  Special  Trustees  for  the  United  Bristol  Hospitals,  for  their  graciousness  and  for 
helping  us  coordinate  the  workshop.  We  would  especially  like  to  recognize  the  effort  and 
work  performed  by  David  Simon,  Fritz  Morgan,  Joni  Ropelewski  and  Jan  Came  who  were 
instmmental  in  putting  together  the  pre-workshop  and  final  reports,  and  coordinating  the 
logistics  of  the  workshop. 

Eastwood  Park  provided  the  perfect  venue  for  participants  to  get  acquainted  both  profession¬ 
ally  and  personally.  It  was  the  spirit  of  the  participants  and  the  relationships  that  were  devel¬ 
oped  during  the  workshop,  that  while  difficult  to  capture  in  print,  were  among  the  most 
important  elements  of  the  workshop's  success. 

We  are  confident  that  the  area  of  Robotics  and  Computer  Assisted  Medical  Interventions  will 
remain  an  exciting  and  expanding  area  with  great  potential  to  improve  the  clinical  practice  of 
medicine. 

Thank  you  and  we  hope  that  you  will  enjoy  reading  the  RCAMI  report.  We  hope  to  see  you  in 
the  United  States  for  a  followup  workshop  in  1998. 

Anthony  M.  DiGioia,  M.D. 

Takeo  Kanade,  Ph.D. 

Peter  Wells,  Ph.D.,  D.Sc. 

Workshop  Coordinators 
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Section  1  -  Executive  Summary 

The  Second  International  Workshop  on  Robotics  and  Computer  Assisted  Medical  Interven¬ 
tions  (RCAMI)  was  held  at  Eastwood  Park  near  Bristol,  England,  June  23-26,  1996.^  The  pri¬ 
mary  goal  of  the  workshop  was  to  bring  physicians  and  researchers  together  to  assess  the 
current  status,  identify  the  future  research  needs  and  opportunities,  and  facilitate  international 
collaboration  and  information  exchange  in  the  field  of  RCAMI.  The  workshop  was  organized 
by  Anthony  M.  DiGioia  III,  Takeo  Kanade,  and  Peter  N.T.  Wells,  with  assistance  from  Fritz 
Morgan  and  David  Simon.  Major  support  was  provided  by  the  National  Science  Foundation 
(NSF),  the  Defense  Advanced  Research  Projects  Agency  (DARPA),  U.S.  Army  Medical 
Research  and  Materiel  Command,  Engineering  and  Physical  Science  Research  Council  (UK), 
various  commercial  partners,  and  was  hosted  by  the  Special  Trustees  for  the  United  Bristol 
Hospitals. 

Participants  were  selected  to  provide  equal  representation  from  leading  physicians  and 
researchers  in  the  RCAMI  field,  as  nominated  by  their  peers.  Representatives  from  govern¬ 
ment  and  industry  were  invited  and  encouraged  to  present  their  agencies’  missions,  and  to 
contribute  in  all  workshop  discussions. 

Unlike  many  other  similar  workshops,  substantive  work  was  put  in  prior  to  the  workshop.  In 
preparation,  participants  were  asked  to  submit  three  to  five  exemplary  papers  within  their  field 
of  research,  and  to  complete  a  questionnaire.  A  preliminary  report  was  written  by  the  organiz¬ 
ers  from  these  responses  to  define  a  starting  point  for  workshop  discussions.  The  report  out¬ 
lined  a  suggested  set  of  issues;  however,  the  workshop  agenda  was  flexible  to  permit 
identification  and  discussion  of  other  relevant  topics  by  the  participants. 

We  have  divided  the  RCAMI  field  into  four  sub-areas: 

1.  Image  Guided  Therapy  -  the  use  of  images  obtained  either  during  or  prior  to  treatment, 
coupled  with  the  use  of  computers,  sensors,  graphics,  or  other  technologies  to  assist  or 
guide  the  administration  of  treatment.  For  the  purpose  of  this  workshop,  this  group  did  not 
consider  active  or  semi-active  robotic  systems,  although  many  robotic  systems  employ 
image  guidance  to  administer  treatment. 

2.  Robotics  -  the  intra-operative  use  of  active  or  semi-active  robotic/manipulation  systems  to 
significantly  enhance  the  ability  of  humans  to  perform  interventional  procedures. 

3.  Surgical  Simulators  -  the  use  of  medical  imaging,  computer  graphics,  biomechanical  anal¬ 
ysis,  and  virtual  environments,  to  simulate  surgery  for  medical  education,  scientific  analy¬ 
sis  and  pre-treatment  planning. 

4.  Teleintervention  -  the  application  of  information-based  technologies  to  deliver  procedural 
health  care  through  an  electronic  interface.  Indirect  patient  contact  is  implicit;  however, 
the  distance  separating  patient  and  physician  may  be  insignificant,  or  great. 

Workshop  participants  were  divided  into  four  groups,  each  concentrating  on  a  particular  appli¬ 
cation  area,  with  each  group  assigned  a  physician  and  researcher  as  leaders.  Roughly  70%  of 
the  workshop  was  devoted  to  group  discussion,  and  the  remaining  time  was  spent  addressing 


1.  The  first  NSF  Workshop  on  Computer- Assisted  Surgery  was  held  February  28  -  March  2,  1993  in 
Washington,  D.C.,  and  was  organized  by  Russell  H.  Taylor  and  George  A.  Bekey. 
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common  issues.  An  important  result  of  each  working  group  was  a  list  of  technical  and  clinical 
challenges  which  must  be  met  in  order  to  advance  the  RCAMI  field.  Common  themes  identified 
from  these  lists  include: 

•  Soft  tissue  modeling  -  the  integration  of  models  incorporating  soft  tissue  characteristics  into 
RCAMI  systems.  Technical  challenges  include:  representation  -  defining  the  computational 
framework;  segmentation  -  delineating  soft  tissue  within  3-D  medical  images;  generation  - 
constructing  the  models;  tracking  -  identifying  soft-tissue  movement  in  real-time  during  sur¬ 
gery;  deformation  -  handling  non-rigid  structures;  registration  -  establishing  correspondence 
between  two  or  more  representations  of  a  soft-tissue  structure;  and  validation  -  ensuring 
model  correctness. 

•  Functional  modeling  -  the  integration  of  physiologic  and  anatomic  data  into  coherent  models. 
This  modeling  enables  exploration  of  functional  consequences  of  a  proposed  intervention, 
examination  of  surgical  options,  optimization  of  techniques,  and  prediction  of  surgical  out¬ 
comes. 

•  Clinical  validation  -  demonstration  of  clinical  benefit,  cost  reduction,  and/or  cost-effective¬ 
ness.  Validation  is  critical  for  the  development,  justification,  and  clinical  acceptance  of 
RCAMI  systems,  yet  no  adequate  measures  of  complex  medical  task  performance  have  been 
developed. 

•  Technical  validation  -  satisfaction  of  technical  requirements.  Key  requirement  areas  include: 
accuracy  of  models,  mechanisms,  algorithms,  imagers,  sensors,  and  complete  systems; 
usability  of  software  and  hardware  interfaces  and  devices;  safety  to  patients,  users  and  devel¬ 
opers;  robustness  of  mechanisms,  algorithms,  and  sensors.  To  minimize  costs  and  avoid 
unnecessary  complexity,  task  requirements  must  be  carefully  defined.  New  evaluation  meth¬ 
ods  should  be  designed  and  applied  to  each  new  RCAMI  system  as  it  is  developed. 

•  Applied  research  -  fundamental  RCAMI  research  problems  should  be  studied  in  a  manner 
which  is  closely  coupled  with  the  development  and  evaluation  of  clinical  prototypes  in  key 
application  areas.  One  area  requiring  significant  work  is  the  intelligent  design  of  human- 
machine  interfaces.  For  example,  if  a  robotic  system  is  to  operate  as  a  true  “assistant”  under  a 
surgeon’s  supervision,  it  needs  the  ability  to  follow  the  surgical  procedure  as  it  progresses,  to 
recognize  organ  systems,  and  to  respond  intelligently  to  the  surgeon’s  high-level  commands. 

During  the  workshop,  overviews  of  each  application  area  were  presented.  There  were  also  intro¬ 
ductions  to  issues  in  safety,  technology  transfer,  government  regulations,  and  establishing  com¬ 
mon  technical  vocabulary.  The  application  overviews  provided  a  framework  for  inter-group 
understanding,  while  the  common  issue  talks  provided  excellent  structure  for  application-spe¬ 
cific  discussions  within  each  group. 

The  workshop  final  report  provides  detailed  information  on  workshop  background,  discussions, 
and  recommendations,  and  should  be  read  by  any  individual  with  an  interest  in  the  RCAMI 
area.  In  particular,  researchers,  physicians,  members  of  funding  agencies,  policy  makers,  admin¬ 
istrators,  and  industry  representatives  will  find  useful  information  in  this  document  which  will 
be  published  in  several  venues,  including  an  electronic  version  on  the  World  Wide  Web 
(http://www.ri.cmu.edu/mrcas/rcami.html).  It  is  recommended  that  a  similar  workshop  be  held 
in  2-3  years  to  re-evaluate  the  state  of  the  RCAMI  field. 
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(Copyright:  David  C.  Rio,  Hon  FMPA  -  Reproduced  with  permission) 


Section  2  -  Report  Overview 

The  RCAMI  workshop  report  is  organized  as  follows.  Sections  3-6  contain  the  outcomes  of 
individual  working  groups.  Each  section  has  a  common  structure: 

•  Executive  Summary  -  A  brief  definition  of  the  application  area,  followed  by  several  key 
proposals  for  future  research. 

•  Definition  -  An  extended  definition  of  the  application  area,  possibly  including  background 
information. 

•  Research  Directions  -  Outlines  future  research  and  clinical  directions  of  this  application 
area. 

•  Review  of  Current  Technology  -  Summarizes  the  state-of-the-art  in  this  application  area. 

•  Technical  and  Research  Issues  -  Describes  the  technical  problems  which  must  be  solved  for 
advancement  of  this  application  area. 

•  Summary  / Recommendations  -  Working  group  conclusions  and  recommendations. 

Small  variations  in  the  above  structure  may  exist  between  working  group  reports,  although 
each  addresses  all  of  the  above  issues. 

The  remainder  of  the  report  contains  supplementary  materials  organized  into  several  appendi¬ 
ces.  Appendix  A  contains  a  list  of  the  workshop  participants.  Appendices  B  through  E  contain 
presentation  materials  from  the  keynote  speeches  in  the  four  application  areas.  Appendices  F 
through  H  contain  materials  from  special  presentations  on  safety,  technology  transfer  and 
image-guided  tumor  diagnosis,  respectively.  Appendix  I  contains  the  working  group  frame¬ 
work  provided  to  the  participants  in  the  pre-workshop  report.  Appendix  J  contains  a  summary 
of  the  questionnaire  responses  compiled  for  the  pre-workshop  report.  Appendix  K  contains  a 
checklist  distributed  to  the  participants  to  stimulate  discussion.  Appendix  L  contains  a  sched¬ 
ule  of  the  workshop  sessions.  Finally,  Appendix  M  contains  a  bibliography  which  was  com¬ 
piled  from  the  list  of  “exemplary”  papers  submitted  by  each  participant  before  the  workshop. 
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Section  3  -  Image-Guided  Therapy 

Chaired  by:  Richard  D.  Bucholz  and  Lutz  R  Nolte 

3.1  Executive  Summary 

Image-Guided  Therapy  -  the  use  of  images  obtained  either  during  or  prior  to  treatment,  cou¬ 
pled  with  the  use  of  computers,  sensors,  graphics,  or  other  technologies  to  assist  or  guide  the 
administration  of  treatment.  For  the  purpose  of  the  workshop,  this  group  did  not  consider 
active  or  semi-active  robotic  systems,  although  many  robotic  systems  employ  image  guidance 
to  administer  treatment. 

The  following  list  represents  the  major  proposed  directions  for  future  research  initiatives  in 
the  area  of  Image-Guided  Therapy: 

♦  Development,  validation  and  evaluation  of  clinical  prototypes  in  key  applications  that 
merge  enabling  technologies.  This  would  include  development  of  modular  components 
such  as  graphical  user  interfaces  (GUIs),  validation  techniques,  and  user-machine  inter¬ 
faces  to  facilitate  new  image-guided  therapy  systems. 

•  Integration  and  characterization  of  soft  tissue  in  image-guided  therapy,  including:  segmen¬ 
tation,  tracking,  modeling,  deformation,  registration,  and  validation. 

The  image-guided  therapy  group  consisted  of  four  neurosurgeons,  two  radiologists,  one  cardi¬ 
ologist,  one  orthopaedist,  one  otorhinolaryngologist,  two  physicists,  five  engineers,  three  com¬ 
puter  scientists,  and  four  individuals  representing  either  corporate  or  governmental  entities. 
This  combination  is  biased  towards  neurosurgeons  and  related  disciplines,  which  influenced 
the  nature  of  the  discussions. 

3.2  Definitions 

This  group’s  definition  of  image-guided  therapy  (IGT)  includes  three  major  components:  a 
therapeutic  object,  a  virtual  object,  and  a  navigational  device  (Figure  3-1).  The  therapeutic 
object  consists  of  the  patient  (or  parts  of  the  patient)  and  an  associated  therapeutic  modality. 
The  virtual  object  is  generated  by  means  of  modem  imaging  or  signal  sensing  and  processing. 
This  includes  diverse  virtual  objects  such  as  endoscopic  images.  The  navigational  device 
enables  precise  therapy  by  utilizing  the  virtual  object  registered  to  the  therapeutic  object. 
Image  guided  therapy  is  based  on  an  estimated  transformation  between  the  coordinate  systems 
of  the  navigational  device,  and  the  therapeutic  and  virtual  objects.  Registration  estimates  this 
transformation.  Calibration  is  typically  required  for  the  navigational  device  and  the  means  of 
virtual  object  generation.  This  definition  can  be  viewed  as  overly  broad,  but  it  must  be  non¬ 
specific  to  incorporate  the  full  spectmm  of  imaging  means  now  clinically  available.  This  defi- 
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Figure  3-1 :  Schematic  of  Image-Guided  Therapy 

nition  also  includes  the  fabrication  through  lithography  of  objects  which  can  be  implanted 
within  the  patient  to  correct  defects,  such  as  the  repair  of  skull  defects  using  computer  gener¬ 
ated  components.  A  particular  image-guided  therapy  may  employ  multiple  navigational 
devices,  or  multiple  therapeutic  and  virtual  objects. 

Within  an  IGT  system,  registration  can  be  performed  using  a  variety  of  data  types,  as  outlined 
in  Figure  3-2.  In  particular,  registration  of  the  patient  to  pre-operative,  intra-operative  and 
post-operative  data  is  possible.  In  order  to  perform  an  IGT,  tools,  effectors  and/or  therapeutic 
modalities  must  be  registered  to  the  patient  as  well. 


8 


Section  3  -  Image-Guided  Therapy 


Figure  3-2:  The  registration  process 

Figure  3-3  shows  a  typical  image-guided  navigational  system  employing  an  optical  digitizer 
(white  horizontal  bar  at  top  of  image)  to  track  surgical  instruments  (in  the  surgeon’s  hand) 
modified  by  the  addition  of  light  emitting  diodes  (LEDs).  In  this  application,  a  cranial  proce¬ 
dure  is  being  performed,  and  the  head  of  the  patient  is  tracked  using  a  black  arc  equipped  with 
LEDs  attached  to  the  patient’s  head  (seen  directly  above  surgeon’s  left  hand).  The  position 
and  orientation  of  the  surgical  instrument  is  displayed  continuously  on  the  large  monitor  on 


Figure  3-3:  An  image-guided  navigational  system  (courtesy  of  Richard  Bucholz). 
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the  left.  Such  a  system  could  be  employed  for  other  specialties  simply  by  modifying  the 
instruments  and  the  device  used  to  track  the  body  part  undergoing  treatment.  Figure  3-4 
shows  a  second  navigational  system  using  similar  components  for  accurately  placing  pedicle 
screws  in  spine  surgery. 


Figure  3-4:  An  image-guided  navigational  system  for  pedicle  screw  insertion  (courtesy  of 
Lutz  Nolte). 

3.3  Research  Directions  and  Review  of  Current  Technology 

Applications  in  which  IGT  could  be  employed  were  extensively  discussed.  Applications  can 
be  classified  by  whether  image  guidance:  1)  enables  the  procedure  to  occur  or  2)  allows  the 
procedure  to  be  performed  either  with  greater  safety,  less  invasiveness,  improved  efficiency,  or 
lower  cost.  Examples  of  procedures  which  can  only  be  performed  by  image  guidance  include 
functional  neurosurgery,  gene  therapy,  radiosurgery,  and  localized  drug  delivery.  Examples  of 
procedures  which  can  benefit  from  image  guidance  include  biopsy,  tumor  resection,  ENT 
sinus  procedures,  joint  reconstructions  and  replacements.  Minimally  invasive  procedures 
often  fall  into  the  second  category,  as  the  application  of  image  guidance  permits  reduced 
exposures  and  tissue  dissection  to  accomplish  the  therapeutic  goal.  Procedures  performed 
under  real-time  imaging,  even  those  as  simple  as  the  use  of  fluoroscopy  in  orthopaedics,  can 
be  viewed  as  image-guided  therapies.  Key  applications  that  are  attractive  in  terms  of  incidence 
of  disease  in  the  population  and  desirability  of  image  guidance  were  identified.  They  are 
grouped  below  by  clinical  application  area. 

It  should  be  emphasized  that  there  exists  a  tremendous  discrepancy  in  the  employment  of 
image  guidance  across  medical  disciplines.  In  neurosurgery,  image  guidance  is  routinely 
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employed  to  permit  functional  interventions  using  framed  stereotaxy,  while  other  specialties 
have  yet  to  develop  initial  applications.  However,  even  in  neurosurgery,  a  critical  component 
of  image  guidance  is  the  concept  of  interactivity,  in  which  the  surgeon  interacts  constantly 
with  the  virtual  object  and  the  navigational  device  during  therapy.  Fully  interactive  systems 
are  just  now  being  approved  by  government  regulatory  bodies  and  are  becoming  common¬ 
place  with  neurosurgery. 

3.3.1  General  surgery 

The  application  of  IGT  to  general  surgery  is  entirely  dependent  on  registration  and  tracking  of 
soft  tissue,  which  is  the  target  of  most  general  surgical  interventions.  Therefore,  the  applica¬ 
tion  to  general  surgery  must  await  the  development  of  techniques  that  can  track  soft  tissue. 
Assuming  that  soft  tissue  can  be  handled,  there  are  significant  applications  in  procedures 
directed  towards  the  liver,  pancreas,  kidneys,  and  pelvic  structures.  Of  these  applications  a 
disease  entity  with  a  particularly  high  incidence  involves  obstruction  of  the  biliary  system. 
Cholecystectomy,  which  has  already  been  improved  by  image  guidance  in  the  form  of  endo¬ 
scopic  intervention,  could  be  further  enhanced  by  tracking  the  soft  tissues  surrounding  the  gall 
bladder  in  real-time,  increasing  the  safety  and  decreasing  the  amount  of  dissection  needed  for 
this  common  procedure.  The  staging  of  metastatic  disease  by  image-guided  minimally  inva¬ 
sive  techniques  would  reduce  the  suffering  of  patients  with  advanced  cancer  by  reducing  inva¬ 
siveness. 

Trauma  to  the  abdomen  and  pelvis  is  rampant  in  the  inner  city  population  of  the  United  States. 
Although  CT  scanning  has  advanced  the  management  of  abdominal  trauma  by  detecting  the 
presence  of  a  perforation  of  a  viscus,  surgery  for  such  an  injury  is  carried  out  by  traditional 
techniques  in  which  the  entire  digestive  system  is  exposed  and  manually  inspected  for  tears. 
The  development  of  imaging  technologies  which  could  reliably  identify  the  location  of  a  vis¬ 
cus  tear,  coupled  to  the  surgical  act  through  IGT,  would  have  a  tremendous  impact  on  reduc¬ 
tion  of  length  of  surgery  (a  critical  concern  in  these  medically  unstable  patients).  IGT  holds 
the  promise  of  focusing  treatment  upon  injured  organs  to  repair  the  damage  while  sparing 
other  organs  from  trauma  caused  by  the  surgical  intervention. 

Finally,  breast  cancer  comprises  one  of  the  leading  sources  of  early  death  in  the  female  popu¬ 
lation.  IGT  has  been  applied  to  the  breast  with  robotic  biopsy  devices.  However,  the  advent  of 
soft  tissue  tracking,  registration,  and  modeling  would  enable  fast,  easy,  and  reliable  breast 
biopsy  with  minimal  discomfort  to  the  patient,  allowing  for  the  rapid  management  of  breast 
cancer. 

3.3.2  Neurosurgery 

Neurosurgery  has  a  long  list  of  procedures  which  are  only  possible  using  image  guidance. 
Most  of  these  procedures  are  grouped  under  the  subspecialty  created  for  this  purpose,  stereo- 
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tactic  surgery.  This  field  within  neurosurgery  has  seen  unparalleled  growth  fueled  by  tremen¬ 
dous  advances  in  neurological  imaging.  Standard  procedures  within  neurosurgery  that  rely  on 
image  guidance  include:  pallidotomy  for  Parkinson’s  Disease,  biopsy  of  intracranial  lesions, 
treatment  of  the  same  using  focused  irradiation  (stereotactic  radiosurgery),  resection  of  intrin¬ 
sic  gliomas  using  frameless  stereotaxis,  and  procedures  upon  the  brainstem. 

Image  guidance  has  become  so  accepted  within  neurosurgery  that  most  surgeons  anticipate 
that  all  intracranial  procedures  will  have  some  component  of  image  guidance  within  the  next 
five  years.  Key  applications  within  neurosurgery  will  evolve  around  resections  of  lesions 
within  eloquent  cortex  using  real-time  or  pre-operative  functional  imaging  to  differentiate  crit¬ 
ical  normal  tissue  from  malignancies.  Intractable  epilepsy,  with  50,000  new  cases  yearly 
within  the  United  States,  would  become  correctable  by  surgical  resection  if  precise  functional 
imaging  is  developed  to  detect  abnormal  tissue.  Further,  as  new  effectors  are  developed  within 
neurosurgery,  such  as  gene  therapy  for  tumors,  the  delivery  of  these  agents  must  be  guided  by 
high  resolution,  high  definition  images  demonstrating  the  exact  location  of  abnormal  tissue. 

Finally,  image  guidance  will  be  applied  to  the  spine,  with  emphasis  on  the  proper  positioning 
of  spinal  instrumentation  (e.g.,  pedicle  screws).  Although  image  guidance  is  not  enabling  in 
this  application,  it  can  significantly  reduce  the  complications  of  spinal  instrumentation,  which 
is  increasingly  employed  to  treat  degenerative  spine  disease  of  our  aging  population. 

3.3.3  Urology 

Lesions  of  the  prostate  are  an  enormous  clinical  problem.  It  has  been  estimated  that  1/3  of  all 
men  might  benefit  from  a  procedure  on  the  prostate  in  their  lifetime.  Transurethral  resection  of 
the  prostate  (TURP),  currently  performed  via  an  endoscope,  has  a  significant  failure  rate  of 
1.5  to  1.8%  yearly,  requiring  patients  to  undergo  repeated  operations.  This  failure  rate  is 
directly  related  to  the  amount  of  hypertrophied  gland  remaining;  many  studies  have  indicated 
that  during  a  procedure  only  38%  of  the  gland  on  average  is  removed.  More  radical  attempts 
to  resect  the  prostate  may  result  in  perforation  of  the  prostate  capsule,  which  can  lead  to 
increased  incidence  of  impotence  postoperatively.  The  use  of  image  guidance  to  track  a 
resecting  tool  to  ensure  maximal  resection  of  the  prostate  would  have  a  dramatic  impact  on  the 
failure  and  complication  rate,  and  would  be  dependent  on  the  development  of  soft  tissue  track¬ 
ing  and  modeling  methods. 

Prostatic  carcinoma  is  another  urological  condition  resulting  in  tremendous  disability  and 
mortality.  Many  patients  with  tumor  metastatic  to  the  spine  die  paralyzed  and  incontinent,  as 
these  lesions  tend  to  be  resistant  to  palliative  radiotherapy.  The  application  of  image  guidance, 
so  that  radiation  therapy  could  be  focused  upon  the  tumor  and  avoid  the  spinal  cord,  would 
relieve  the  suffering  and  deformity  associated  with  this  condition. 
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3.3.4  Cardiovascular  Surgery 

Coronary  vascular  disease  is  one  of  the  major  causes  of  death  within  our  population.  Three 
areas  in  which  IGT  could  have  an  impact  include:  dilatation  of  the  coronary  arteries  by 
intraluminal  image-guided  placement  of  vascular  stents,  resection  of  ventricular  aneurysms 
guided  by  real-time  imaging  of  the  coronary  wall  to  differentiate  functional  versus  nonfunc¬ 
tional  myocardium,  and  correction  of  cardiac  arrhythmias  by  atrial  interventions  guided  by 
intraprocedural  sensing  of  abnormal  cardiac  conductivity.  The  effective  treatment  of  these  dis¬ 
eases  using  functional  rather  than  purely  image  based  data  underscores  the  need  to  integrate 
all  forms  of  data  for  IGT.  The  arrhythmia  application  relies  upon  the  development  of  three 
dimensional  models  of  cardiac  transmission,  and  the  development  of  miniaturized  effectors 
which  could  block  these  pathways  with  high  precision.  Such  applications  could  effectively 
treat  atrial  fibrillation,  a  common  cardiac  arrhythmia  causing  significant  disability  with  the 
elderly. 

3.3.5  Otorhinolaryngology 

Sinus  surgery  is  extremely  common  within  our  society,  and  in  the  majority  of  cases  can  be 
handled  without  image  guidance  (other  than  that  provided  by  a  nasal  endoscope).  However, 
there  is  an  appreciable  risk  of  inadvertent  penetration  of  the  cranial  vault  especially  in  cases  in 
which  the  normal  anatomy  has  been  altered  either  through  prior  surgery  or  though  disease 
progression.  IGT  could  enhance  the  safety  of  such  procedures  by  indicating  the  position  of 
instruments  within  the  nasal  cavity  and  alerting  the  surgeon  as  the  floor  of  the  cranial  vault  is 
approached.  A  prime  example  of  the  fusion  of  robotic  technology  with  IGT  would  be  the 
attachment  of  a  passive  robot  to  surgical  instruments  allowing  surgery  only  within  the  con¬ 
fines  of  nasal  sinuses.  Once  the  image-guided  robot  detected  movement  of  an  instrument  out¬ 
side  of  the  sinus,  it  would  become  active,  preventing  further  insertion  of  the  instrument.  This 
paradigm  could  be  useful  in  other  surgical  procedures  as  well. 

3.3.6  Orthopaedic  Surgery 

Orthopaedic  surgery  deals  with  degenerative,  traumatic,  and  congenital  disease  of  the  locomo¬ 
tor  apparatus.  Therefore,  the  primary  potential  of  IGT  in  this  area  focuses  on  therapeutic 
actions  on  bone  rather  than  soft  tissue.  Example  applications  for  which  IGT  augments  existing 
capabilities  are  the  insertion  of  pedicle  screws  or  the  functional  placement  of  the  aeetabular 
cup  in  total  hip  replacement  surgery.  An  example  application  in  which  IGT  enables  new  pro¬ 
cedures  is  surgical  navigation  combined  with  image  fusion  for  spinal  cage  delivery.  IGT  in 
orthopaedics  (as  in  other  sub-specialties)  is  the  key  for  combining  advanced  diagnosis, 
pre-operative  planning,  intra-operative  tool  actions,  corrective  procedures  and  postoperative 
evaluation.  Examples  are  femoral  or  acetabular  osteotomies.  By  incorporating  biomechanical 
methods  during  pre-operative  planning  and  intra-operative  virtual  object  updating,  significant 
improvements  are  possible  via  the  optimization  of  biomechanical  parameters. 
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Short  term  applications  of  IGT  include  the  integration  of  IGT  in  conventional  approaches  for 
hip  and  knee  replacement,  and  surgical  interventions  to  cure  low  back  pain,  areas  of  over¬ 
whelming  socio-economic  importance.  In  the  long  term,  IGT  may  have  a  significant  impact  on 
fracture  fixation,  if  particularly  fast,  robust,  and  easy  to  handle  systems  become  available. 
However,  the  dominant  long  term  goal  would  be  the  development  of  minimally  invasive  tech¬ 
niques  involving  the  delivery  of  novel  implants  and  growth  factor  carriers. 

3.4  Technical  and  Research  Issues 

3.4.1  Accuracy 

Accuracy  is  perceived  to  be  a  major  limiting  factor  to  the  broad  application  of  image  guid¬ 
ance.  Accuracy  in  image  guidance  can  be  classified  into  three  components:  mechanical  accu¬ 
racy,  application  accuracy,  and  operational  accuracy.  Mechanical  accuracy  refers  to  the 
accuracy  of  the  navigational  devices;  application  accuracy  refers  to  mechanical  accuracy,  the 
accuracy  of  the  process  by  which  the  virtual  object  was  obtained,  and  the  accuracy  of  the  reg¬ 
istration  process.  Operational  accuracy  is  application  accuracy  combined  with  errors  intro¬ 
duced  during  the  intervention.  For  example,  tissue  deformation  during  surgery  markedly 
increases  the  inaccuracy  of  a  procedure  guided  solely  upon  pre-operative  imaging.  For  neuro¬ 
surgical  applications,  the  IGT  working  group  concluded  that  mechanical  and  application  accu¬ 
racy  of  current  systems  are  adequate  for  the  majority  of  interventions,  while  operational 
accuracy  is  the  key  issue  to  be  resolved.  This  conclusion  is  not  necessarily  applicable  to  other 
medical  disciplines  such  as  orthopaedics,  for  which  existing  registration  techniques  may  not 
guarantee  sufficient  application  accuracy.  For  all  fields,  increased  accuracy  is  accompanied  by 
increased  cost  and  system  complexity.  Therefore,  for  any  given  application,  the  required  accu¬ 
racy  should  be  carefully  defined  in  order  to  minimize  cost  and  avoid  unnecessary  complexity, 
while  satisfying  task  requirements. 

The  consensus  of  the  group  is  that  operational  accuracy  in  IGT  requires  improvement.  Interac¬ 
tive  real-time  update  of  the  virtual  object  during  the  procedure  was  identified  as  a  key  method 
for  reducing  operational  errors.  Conceptually,  there  are  two  ways  to  update  the  virtual  object: 
repeated  intra-operative  imaging,  and  modeling  of  tissue  behavior  when  subjected  to  therapy. 
Physical  modeling  may  require  tactile  sensory  feedback  for  predicting  deformations.  Intra¬ 
operative  imaging  can  be  performed  using  modalities  such  as  MRI,  CT,  ultrasound,  or  fluoros¬ 
copy.  For  virtual  object  updating,  the  resolution  of  intra-operative  imaging  does  not  have  to 
match  that  of  the  pre-operative  imaging.  Intra-operative  ultrasound  was  mentioned  as  a  tech¬ 
nology  which  has  desirable  cost,  availability,  and  accuracy  characteristics.  Neither  method  of 
virtual  object  updating  has  been  sufficiently  developed  to  allow  for  routine  clinical  use.  The 
problem  of  modeling  non-rigid  therapeutic  objects  and  the  associated  response  to  therapy  is 
present  throughout  many  medical  disciplines,  and  is  a  key  area  for  future  research. 
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3.4.2  Other  Issues 

Several  problems  must  be  solved  before  widespread  adoption  of  IGT  is  possible.  Two  of  the 
most  important  are  the  level  of  human  interaction  required,  and  the  ease  with  which  IGT  can 
be  seamlessly  integrated  into  the  clinical  environment.  For  IGT  to  be  practical,  medical 
images  must  be  obtained  quickly,  and  transmitted  easily  to  the  site  of  the  intervention.  Image 
data  manipulation  such  as  segmentation  or  multimodality  fusion  should  be  a  standard  func¬ 
tion,  and  should  be  automated  whenever  possible.  User  interfaces  for  navigational  devices, 
registration,  and  pre-operative  planners  should  be  simplified  and  standardized  over  diverse 
applications.  In  particular,  user  interaction  during  the  registration  process  is  a  potentially  trou¬ 
blesome  area.  Preferably,  registration  and  associated  data  acquisition  should  be  automated, 
such  as  in  spinal  registration  using  intra-operative  planar  ultrasonic  or  fluoroscopic  images. 
However,  in  every  case,  the  registration  result  should  be  verified  by  the  surgeon.  This  may  be 
technically  difficult  in  closed  (i.e.,  minimally  invasive)  procedures,  and  is  an  area  for  future 
research. 

One  concept  discussed  by  the  group  was  the  development  of  robust  middleware  (i.e.,  standard 
IGT  interfaces  and  components  on  which  surgeons  can  be  trained  and  use  for  a  variety  of  pro¬ 
cedures).  Middleware  would  allow  for  plug-in  modules  to  incorporate  a  variety  of  different 
functionalities  (e.g.,  ability  to  interchange  imaging  modalities,  input  devices,  algorithmic 
components,  navigational  devices,  etc.). 

Above  all,  widespread  acceptance  of  IGT  requires  the  demonstration  of  medical  benefit, 
reduced  cost,  or  both.  The  IGT  group  perceived  that  such  a  demonstration  will  be  difficult  to 
perform,  and  will  be  subject  to  controversy  over  the  analysis  used.  Nevertheless,  significant 
funding  should  be  allocated  to  this  important  endeavor. 

3.4.3  Enumeration  of  Technical  Challenges 

Table  3-1  contains  an  overview  of  the  technical  issues  and  challenges  which  must  be  solved 
before  widespread  adoption  of  IGT  is  possible.  Each  cell  in  the  table  represents  issues  related 
to  a  particular  IGT  functionality.  Superscripts  attached  to  particular  issues  indicate  the  time- 
frame  during  which  it  is  expected  that  significant  research  will  address  the  issue. 

3.5  Summary  /  Recommendations 

In  conclusion,  the  group  identified  two  key  areas  towards  which  research  effort  and  funding 
should  be  foeused.  By  focusing  on  these  concepts,  the  effectiveness  of  limited  research  dollars 
could  be  greatly  enhanced. 

The  first  area  is  the  development,  validation,  and  evaluation  of  elinical  prototypes  in  key 
applications  that  merge  enabling  technologies.  This  would  include  development  of  modular 
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Section  3  -  Image-Guided  Therapy 


components,  such  as  graphical  user  interfaces,  system  and  component  validation,  and  human- 
machine  interfaces  to  facilitate  new  image-guided  therapies.  For  example,  with  modular  com¬ 
ponents,  a  system  developed  for  cranial  applications  could  be  modified  to  track  vertebral  bod¬ 
ies  during  spinal  fusion.  This  modification  would  require  development  of  instrumentation 
appropriate  for  the  spine,  and  a  method  to  track  the  vertebral  body.  Minor  modifications  to  the 
user  interface  would  allow  tracking  of  pedicle  screws  as  they  were  inserted  into  the  spine. 
With  modular  components,  clinical  feedback  from  one  application  can  be  used  in  other  areas. 
In  this  way,  new  systems  do  not  have  to  be  developed  from  the  ground  up,  and  as  new  devices 
are  developed  (e.g.,  digitizers,  microscopes,  endoscopes)  all  applications  can  benefit. 

The  second  area  of  research  focus  is  the  integration  and  characterization  of  soft  tissue  during 
image-guided  therapy.  This  would  include  segmentation,  tracking,  modeling,  deformation, 
registration,  and  validation  of  models.  Subspecialities  dealing  with  rigid  structures  (e.g.,  neu¬ 
rosurgery  and  orthopaedics)  benefit  from  the  ease  of  modeling  and  interacting  with  these 
structures.  The  absence  of  image-guided  therapies  for  soft  tissue  procedures  (e.g.,  abdominal 
surgery)  is  a  direct  result  of  the  deformability  of  the  associated  structures  and  the  difficulty 
associated  with  tracking  these  structures  in  real-time.  As  the  ability  to  model  and  track  soft- 
tissue  improves,  new  applications  will  be  developed  in  these  areas. 
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Chaired  by:  Russell  Taylor  and  David  Stulberg 

4.1  Executive  Summary 

Medical  Robotics  -  the  intra-operative  use  of  active  or  semi-active  robotic/manipulation  sys¬ 
tems  to  significantly  enhance  the  ability  of  humans  to  perform  interventional  procedures. 

The  following  list  represents  the  major  proposed  directions  for  future  research  initiatives  in 
the  area  of  Robotics. 

•  Emphasize  development  and  validation  of  prototype  systems  targeted  at  specific  applica¬ 
tions  that  require  significant  advances  in  underlying  component  technologies  such  as  sens¬ 
ing,  manipulation,  human-machine  interaction,  safety,  model  registration,  etc.  Research 
programs  should  be  structured  to  encourage  close  and  continuing  teamwork  between  clini¬ 
cal  end  users,  engineering  researchers,  and  health  economists. 

•  Provide  means  to  facilitate  the  bridge  from  initial  feasibility  prototypes  to  systems  that  can 
be  used  as  effective  research  and  evaluation  platforms  and  to  promote  more  effective  shar¬ 
ing  of  systems  and  technology  infrastructure  between  researchers. 

•  Better  means  for  evaluating  computer-assisted  surgical  techniques  must  be  developed. 
Many  current  clinical  outcome  measurement  tools  are  inappropriate  for  the  accurate  evalu¬ 
ation  of  the  efficacy  of  computer-assisted  surgical  technology. 

4.2  Definitions 

This  group  focused  on  computer-controlled  manipulation  devices  and  their  application  in  sur¬ 
gery.  There  are  two  key  roles  for  robotic  systems  in  medicine: 

•  To  optimize  and  extend  the  use  of  traditional  human  surgical  skills  in  patient  environments 
made  accessible  through  the  use  of  new  technologies,  such  as  endoscopic  cameras.  In  addi¬ 
tion,  robotic  technology  is  itself  crucial  in  extending  the  applicability  of  minimally  invasive 
and  micro  surgical  techniques. 

•  To  provide  functional  abilities  that  human  surgeons  lack.  In  particular,  robotic  systems  can 
provide  a  crucial  link  between  computer-based  pre-operative  planning  and  effective  deliv¬ 
ery  of  the  planned  therapy. 

Humans  and  machines  have  complementary  strengths  and  limitations,  and  the  overall  goal  is 
to  find  ways  to  use  them  together  to  provide  better  and  more  cost-effective  care  than  can  be 
provided  by  either  alone.  Recognized  advantages  of  robotic  systems  include: 

•  The  ability  to  accurately  position  and  reposition  surgical  tools. 
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•  The  ability  to  apply  precisely  calibrated  forces. 

•  The  potential  for  reduction  in  tremor  as  compared  to  human  hands. 

•  The  ability  to  scale  the  magnitudes  of  forces  and  motions  to  be  either  larger  or  smaller  than 
are  possible  by  humans. 

•  The  ability  to  provide  a  stable  platform  for  supporting  and  positioning  surgical  sensors, 
cameras,  or  instruments  in  a  tireless  manner. 


Table  4-1:  Complementary  Capabilities 


Strengths 

Limitations 

Humans 

•  Good  judgment 

•  Strong  hand-eye  coordination 

•  Integrate  extensive  &  diverse  infor¬ 
mation 

•  Very  flexible  and  adaptable 

•  Very  dexterous  at  “human”  scale 

•  Able  to  use  qualitative  information 

•  Superb  hand-eye  coordination 

•  Highly  evolved 

•  Easy  to  instruct  (except  teen-agers) 

•  Explain  themselves  (ditto) 

•  Tremor 

•  Fatigue 

•  Limited  manipulation  ability  & 
dexterity  outside  natural  scale 

•  Bulky 

•  Geometric  accuracy  limited 

•  Do  not  use  quantitative  information 
naturally 

•  Hard  to  keep  sterile 

•  Susceptible  to  radiation,  infection 

Robots 

•  Good  geometric  accuracy 

•  Untiring  &  stable 

•  Potentially  constructed  in  many 
sizes  &  immune  to  infection 

•  Potentially  not  affected  by  radiation 

•  Able  to  incorporate  many  sensors 
(chemical,  force,  acoustic,  etc.)  into 
control  laws 

•  Poor  judgment 

•  Expensive 

•  Technology  is  evolving 

•  Difficult  to  instruct 

•  Inscrutable 

•  Limited  ability  to  do  complex  con¬ 
trol  &  hand-eye  tasks 

4.3  Review  of  Current  Technology:  Existing  and  Emerging  Applications 

A  number  of  robotic  devices  have  been  developed  for  surgical  use:  assistive  devices,  naviga¬ 
tional  aids,  positioning  aids,  and  path  following  robots. 

Robotic  assistive  devices  have  been  developed.  These  devices  aim  to  provide  cost-efficient, 
stable  control  of  surgical  tools  for  tasks  traditionally  performed  by  surgical  interns  or  other 
operating  room  personnel  whose  main  job  is  to  help  the  surgeon.  The  primary  justifications 
for  such  systems  include  (a)  cost  savings  by  reducing  the  number  of  people  in  the  operating 
room,  (b)  improved  access  to  the  patient,  and  (c)  reduction  in  problems  associated  with  human 
fatigue  and  inattention.  Typical  examples  include: 

•  retractors 

•  endoscopic  camera  holders 

•  body  part  positioners 
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•  needle  holders 

•  etc. 

The  primary  limitations  of  currently  available  devices  are:  1)  inability  to  respond  in  a  user 
friendly,  efficient  manner  to  surgeons’  current  and  anticipated  needs;  2)  size;  and  3)  lack  of 
versatility.  These  systems  are  often  cheaper  and  sometimes  better  at  particular  tasks  than  their 
human  counterparts,  but  they  are  by  no  means  as  user  friendly  or  flexible  in  what  they  can  do. 

A  somewhat  related  class  of  emerging  systems  are  those  that  extend  a  surgeon’s  manipulation 
capabilities,  typically  by  permitting  very  accurately  controlled  small  motions  (e.g.,  for  micro¬ 
surgery)  or  by  providing  high  degrees  of  dexterity  within  a  confined  space  (e.g.,  for  laparo¬ 
scopic  surgery).  Such  systems  are  typically  teleoperated,  although  other  control  modes  are 
possible,  and  are  also  discussed  to  some  extent  in  the  Teleinterventions  section  of  this  report 
(see  Section  Section  6  -).  These  systems  have  many  uses  even  when  the  surgeon  and  robot  are 
both  within  the  same  OR.  They  may  also  be  used  in  “shared  autonomy”  modes  to  perform  pre¬ 
cise  positioning  and  path  control  functions  such  as  those  below.  Both  the  basic  functional 
technology  (sensing,  actuation,  control)  and  human-machine  interfaces  for  these  systems  rep¬ 
resent  important  research  challenges.  At  one  level,  surgeons  would  really  like  the  “nostalgic” 
feel  of  traditional  open  surgery  while  performing  tasks  requiring  super-human  precision, 
steadiness,  or  access  to  the  patient’s  anatomy.  At  another  level,  there  is  a  great  opportunity  to 
supplement  such  capabilities  with  additional  unique  functions  based  on  pre-operative  images 
or  intra-operative  sensing. 

Robotic  devices  to  aid  navigation  have  been  and  are  being  developed.  Generally,  the  primary 
advantage  provided  by  such  systems  (often,  rather  over-broadly  referred  to  as  “computer- 
assisted  surgery”  systems)  is  accurate  information  of  the  position  of  surgical  instruments  rela¬ 
tive  to  a  patient’s  anatomy,  as  reflected  in  medical  images.  Although  many  of  the  issues  asso¬ 
ciated  with  such  systems  are  covered  in  the  Image-Guided  Therapy  section  of  this  report  (see 
Section  Section  3  -),  there  is  strong  synergy  with  “robotics”.  First,  many  of  the  key  technolo¬ 
gies  (e.g.,  sensing,  image-to-reality  registration)  associated  with  navigation  are  also  crucial  to 
more  active  robotic  applications,  and  navigational  systems  are  often  used  in  conjunction  with 
such  systems.  Second,  active  robotic  devices  can  be  used  fruitfully  as  components  in  systems 
whose  main  function  is  navigation  or  information  assistance  (e.g.,  positioning  of  imaging 
devices  or  surgieal  microscopes).  One  of  the  main  challenges  for  such  systems  is  better  inte¬ 
gration  with  other  operating  room  equipment  and  with  pre-surgical  planning. 

A  variety  of  systems  have  been  and  are  being  developed  to  position  a  surgical  tool  accurately 
and  safely  relative  to  the  patient’s  anatomy.  Examples  include: 

•  inserting  a  needle  into  a  calyx  of  the  kidney 

•  stereotactic  biopsy  (brain,  breast,  etc.) 

•  percutaneous  pattern  therapy  (brain,  prostate,  liver) 


21 


Section  4  -  Robotics 


•  drill  guides  (spine,  skull,  hip) 

•  total  knee  replacement  guidance  systems 

•  craniofacial  surgery  augmentation  systems 

Generally,  the  target  position  is  determined  from  pre-operative  3D  images  (CT,  MRI,  etc.)  or 
interactively  from  intra-operative  2D  images  (ultrasound,  biplanar  fluoroscopy).  For  cases  in 
which  pre-operative  images  are  used,  a  number  of  techniques  can  be  used  to  register  the  pre¬ 
operative  data  with  intra-operative  reality. 

The  main  limitations  with  current  positioning  systems  are:  lack  of  sensitivity  to  soft  tissues, 
inability  to  respond  to  changes  in  tissue  character  and  motion;  and  limited  ability  to  provide 
real-time  intra-operative  feedback  to  supervising  clinical  staff. 

A  number  of  systems  have  been  created  which  generate  a  path  through  tissue.  The  advantage 
of  these  devices  is  the  accuracy  and  reproducibility  with  which  such  paths  can  be  made. 
Examples  include: 

•  total  hip  replacement  surgery. 

•  transurethral  resection  of  the  prostate. 

•  laser  resection  of  brain  tumors. 

•  high  intensity  focused  ultrasound  (HIFU)  tissue  ablation  under  real  time  MRI  guid¬ 
ance. 

Although  there  are  exceptions,  the  main  limitations  of  this  group  of  robotic  devices  include: 
1)  cost;  2)  size;  3)  degree  of  integration  of  pre-operative  and  intra-operative  planning; 
4)  registration  methods;  5)  lack  of  versatile  end  effectors;  and  6)  lack  of  versatility  in  general. 

The  next  several  pages  contain  images  of  several  existing  surgical  robotic  systems. 
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Figure  4-1:  Robot  systems  for  laparoscopic  surgery.  A  number  of  assistive  systems  have 

been  developed  for  laparoscopic  surgery.  The  clinically  deployed  system  on  the 
top  (AESOP,  developed  by  Computer  Motion,  Inc.)  positions  a  laparoscopic 
camera  under  surgeon  joystick  or  foot  pedal  control.  The  experimental  system 
on  the  bottom  (developed  by  Prof.  Taylor  at  Johns  Hopkins  University) 
performs  similar  functions  using  a  joystick-like  device  clipped  to  the  surgeon’s 
instruments  and  also  includes  a  number  of  autonomous  positioning  capabilities, 
such  as  the  ability  to  position  a  surgical  instrument  on  a  target  designated  by  the 
surgeon  in  video  images. 
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Figure  4-2:  Several  groups  are  investigating  the  use  of  force  compliant  active  robots  in 

various  shared  autonomy  modes  to  extend  human  manipulation  capabilities.  In 
the  scene  above,  the  surgeon  is  manipulating  a  neuroendoscope  for  evacuation 
of  hematomas  (developed  at  Johns  Hopkins  University). 


24 


Section  4  -  Robotics 


Figure  4-3:  Typical  robotic  positioning  applications.  In  the  clinically  applied  system  on  the 
top  (IGOR  system  developed  by  Stephane  Lavallee),  a  robot  is  used  to  position 
a  needle  guide  for  stereotactic  brain  biopsies.  Such  applications  rely  crucially 
on  the  skull  to  provide  a  fixed  frame  of  reference.  In  the  experimental  system  on 
the  bottom  (developed  by  Profs.  Anderson  and  Taylor  at  Johns  Hopkins)  the 
goal  is  to  use  the  robot  to  place  radiation  pellets  and  other  patterns  of  localized 
therapy  under  intraoperative  biplanar  fluoroscopic  guidance  into  abdominal 
organs  such  as  the  liver  and  kidney.  Significant  advances  in  imaging,  model 
registration,  soft  tissue  modeling,  and  robotic  systems  will  be  needed  to  make 
such  systems  routinely  usable.  At  the  same  time,  robotic  systems  offer  a 
potentially  crucial  advantage  of  being  able  to  adapt  quickly  to  achieve  accurate 
placement  in  the  presence  of  patient  respiration  and  soft  tissue  deformation. 
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Figure  4-4:  Path  systems.  In  the  picture  on  the  top,  a  specialized  robotic  device  is  being  used 
for  transurethral  prostate  resection  (developed  by  Prof.  Davies  et  al,  Imperial 
College  and  Guys  Hospital,  UK).  The  picture  on  the  bottom  shows  a  somewhat 
more  versatile  robot  being  used  for  cementless  total  hip  replacement  surgery 
(Robodoc  developed  by  Integrated  Surgical  Systems). 
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4.4  Technical  and  Research  Issues 

4.4.1  Device  Technology 

Advances  are  needed  in  sensors,  actuators,  and  mechanisms  for  tool  positioning  and  tissue 
interaction.  For  minimally  invasive  surgery,  it  is  necessary  to  develop  systems  that  provide 
high  degrees  of  dexterity  in  compact  spaces  ranging  in  size  from  1-2  cm  (intra-abdominal  sur¬ 
gery)  down  to  1mm  or  less  (e.g.,  for  intravascular  surgery)  and  for  positioning  them  accurately 
both  with  “conventional”  manipulators  supporting  instruments  passed  into  the  patient’s  body 
through  small  portals,  and  by  means  of  devices  that  move  through  the  patient’s  body  (e.g., 
flexible  endoscopes,  catheters,  semi-autonomous  “crawlers”).  Further  advances  are  also 
needed  for  “conventional”  microsurgical  systems  for  such  applications  as  eye  surgery  and 
microvascular  surgery.  There  is  also  significant  need  to  integrate  a  variety  of  sensors  into  sur¬ 
gical  end-effectors  and  to  integrate  the  information  into  the  several  levels  of  the  system  con¬ 
trol  hierarchy.  One  example  of  this  would  be  integration  of  pressure  sensors  into  tumor 
injection  and  biopsy  devices.  Another  would  be  means  to  sense  the  hardness  of  bone  in  an 
orthopaedic  bone  machining  application.  This  information  would  be  used  both  to  help  control 
the  tissue  interaction  process  and  as  an  input  for  updating  registration  of  preoperative  and 
intra-operative  models. 

4.4.2  Human-Machine  Interaction 

Better  technology  and  methods  are  needed  to  support  several  forms  of  interaction  between  the 
human  clinician  and  the  computers  controlling  the  robotic  system.  For  direct  telesurgical  con¬ 
trol,  high  bandwidth,  high  dexterity  masters  need  to  be  developed  that  give  the  surgeon  easy 
control  of  a  variety  of  surgical  devices  and  end-effectors  and  that  provide  suitable  propriocep¬ 
tive  and  haptic  feedback  (see  Section  6.0  on  Teleinterventions).  For  supervisory  control  of 
assistive  and  precise  surgical  devices,  natural  methods  for  communicating  the  surgeon’s  inten¬ 
tions  without  imposing  an  undue  burden  on  the  surgeon’s  attention  must  be  developed.  Com¬ 
munication  with  a  robotic  assistant  should  be  no  more  difficult  than  communicating  with  a 
human  assistant.  Indeed,  the  robotic  system  should  be  able  to  “understand”  and  (in  some 
cases)  anticipate  the  surgeon’s  intentions,  by  relating  commands  to  models  of  the  patient  anat¬ 
omy  and  task  plan.  This  capability  will  require  both  significant  advances  in  the  ability  of  sys¬ 
tems  to  model  and  monitor  surgical  procedures  (discussed  below)  and  significant  advances  in 
ways  to  use  such  models  in  conjunction  with  a  variety  of  novel  and  existing  technologies  for 
commanding  and  advising  the  systems. 

It  is  similarly  crucial  that  the  robotic  system  be  able  to  report  its  understanding  of  the  current 
surgical  situation  in  terms  the  surgeon  is  familiar  with.  One  crucial  challenge  will  be  to  find 
ways  to  communicate  the  limitations  (e.g.,  in  registration  accuracy)  of  the  system’s  model 
without  confusing  the  surgeon  to  the  extent  that  the  information  conveyed  is  useless.  Existing 
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techniques  (e.g.,  superimposed  displays)  need  to  be  substantially  improved  and  new  methods, 
not  yet  thought  of,  may  have  to  be  invented. 

In  cases  where  the  robot  is  used  to  augment  the  surgeon’s  manipulation  capabilities,  rather 
than  simply  provide  assistive  functions,  significant  advances  in  telemanipulation  methodology 
may  be  required.  Better  means  must  also  be  developed  for  coupling  human  judgement  with 
machine  consistency  and  stability  in  precise  surgical  manipulation  tasks.  This  will  require  the 
development  of  good  methods  for  the  human  surgeon  and  control  computer  to  share  control  of 
robotic  devices  and  to  “hand-ofP’  control  from  one  to  another  in  a  graceful  manner. 

4.4.3  Integration  with  information  infrastructure 

Improved  methods  for  modeling  patient  anatomy  and  surgical  state  are  crucial,  including: 
methods  for  modeling  deformable  organs  and  structures  based  on  pre-operative  data;  methods 
for  updating  these  models  based  on  real-time  sensing;  and  methods  for  predicting  how  tissue 
will  respond  to  manipulation  forces. 

A  related  crucial  issue  concerns  accuracy.  Accurate  control  of  assistive  manipulation  tasks 
such  as  retraction  or  countertraction  will  be  substantially  improved  by  good  dynamics  models 
of  the  tissues  being  manipulated.  Likewise,  such  tasks  as  accurate  placement  of  needles  into 
solid  organs  and  tumors  may  require  good  means  for  modeling  and  adapting  to  tissue  defor¬ 
mation  during  the  treatment  process.  The  system’s  internal  model  needs  not  only  to  represent 
its  best  current  guess  of  the  patient  geometry,  but  also  possible  modeling  errors.  Application 
software  needs  to  make  use  of  this  information  both  in  planning  and  in  assuring  safe  execution 
of  planned  tasks. 

4.4.4  Safety 

Safety  is  a  crucial  consideration  in  surgical  applications.  Development  of  appropriate  guide¬ 
lines  for  safety,  and  determination  of  which  techniques  are  appropriate  in  particular  situations 
remain  open  areas  for  future  work.  In  addition,  better  methods  for  modeling  intra-operative 
anatomy  and  its  relationship  to  the  robotic  equipment  should  be  developed  for  improving 
patient  safety.  For  example,  safety  can  be  improved  by  identifying  dangerous  regions  and 
requiring  an  explicit  override  from  the  surgeon  before  permitting  a  surgical  cutter  or  instru¬ 
ment  to  enter  them.  Similarly,  models  of  system  registration  error  are  crucial  in  planning  and 
executing  precise  biopsies,  tissue  resections,  and  similar  procedures. 

4.4.5  Integration  into  the  operating  room  environment 

The  successful  introduction  of  robotic  systems  in  the  near  future  requires  that  the  systems  be 
compatible  with  current  operating  room  environments.  However,  truly  complex,  interactive 
robotic  systems  will  require  significant  reconfigurations  of  the  current  operating  room  con¬ 
cept.  This  reconfiguration  is  essential  to  the  successful  integration  of  planning,  registration. 
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tooling  and  evaluation  technologies.  The  sooner  that  computer-assisted  surgery  compatible 
operating  room  (CASCOR)  concepts  are  conceived  and  developed,  the  sooner  medically  spe¬ 
cific  robotic  technologies  can  be  introduced  into  the  operating  room. 

In  addition  to  providing  a  user  friendly  environment  for  robotic  devices,  CASCORs  will  have 
to  address  such  issues  as  sterilization  of  microsensors  and  other  robotic  devices,  the  poten¬ 
tially  unique  electrical  requirements  of  complex  computer  technology,  and  the  ability  to  tran¬ 
sition  an  operating  room  between  computer-assisted  and  conventional  surgical  procedures. 

4.4.6  Evaluation  and  Assessment 

Robotic  technologies  will  and  should  be  critically  evaluated  in  terms  of  cost-effectiveness. 
Many  current  clinical  outcome  measurement  tools  are  inappropriate  for  the  accurate  evalua¬ 
tion  of  the  efficacy  of  computer-assisted  surgical  technology.  There  is  the  potential  to  create 
methods  for  evaluating  the  efficacy  of  computer-assisted  surgical  techniques  (e.g.,  finite  ele¬ 
ment  modeling,  microvascular  flow  rates,  implant  positions).  New  and  appropriate  evaluation 
methods  should  be  designed  and  applied  to  each  new  computer-assisted  surgical  application 
as  it  is  conceived  and  developed. 

4.5  Summary  /  Recommendations 

1 .  Research  model  -  Emphasize  development  and  validation  of  prototype  systems  targeted  at 
specific  applications  that  require  significant  advances  in  underlying  component  technolo¬ 
gies  such  as  sensing,  manipulation,  human-machine  interaction,  safety,  model  registration, 
etc.  In  developing  such  systems,  active  teamwork  between  clinicians  and  technologists  is 
crucial  at  all  stages  to  identify  and  evaluate  key  advantages  from  the  application  of  robotic 
technology  to  the  clinical  problem  being  studied.  Similarly,  issues  of  cost-effectiveness 
and  eventual  deployment  should  be  understood  from  an  early  stage,  and  early  participation 
by  industry  should  be  encouraged.  Great  attention  should  be  paid  to  early  in-vivo  and  clin¬ 
ical  validation  of  results,  both  to  provide  better  understanding  of  actual  needs  and  as  a 
means  of  strengthening  communication  between  members  of  the  team. 

2.  Enabling  technology  research  -  Research  to  develop  critical  enabling  technologies  and 
techniques  should  be  pursued  aggressively.  Although  this  research  will  most  often  be  best 
pursued  within  the  context  of  an  integrated  system  targeted  at  a  particular  application, 
many  topics  have  very  broad  applicability  and  more  broadly  focused  research  may  some¬ 
times  be  appropriate  so  long  as  the  link  to  eventual  application  is  clearly  understood  and 
proper  validation  of  results  is  possible.  A  few  specific  goals  of  particular  interest  to  our 
working  group  include: 

a)  Significantly  extend  the  ability  of  robots  to  perform  surgical  tasks  such  as  retrac¬ 
tion,  dissection,  or  accurate  placement  of  needles  into  deformable  soft  tissues  or 
organs.  This  will  require: 
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i)  Research  to  determine  mechanical  properties  of  various  soft  tissues  and 
organ  systems,  in  particular  force-displacement-velocity  relationships.  In 
addition  to  being  essential  for  modeling  and  sensing  soft  tissue  deforma¬ 
tions,  this  work  may  provide  insight  into  new  ways  for  interacting  with 
such  tissues. 

ii)  Development  of  means  to  sense  (as  well  as  model)  deformation  of  soft  tis¬ 
sues  during  manipulation,  particularly  organs  without  clear  visible  surface 
landmarks  (e.g.,  kidney,  liver,  lung).  This  will  permit  robots  to  autono¬ 
mously  manipulate  tissue  (dissection,  puncture,  etc.). 

iii)  New  methods  for  integrating  models  and  sensory  information  to  perform 
specific  surgical  tasks. 

b)  Develop  new  robotic  manipulators  for  specific  surgical  tasks  and  contexts.  For 
example: 

i)  For  minimally  invasive  surgery,  develop  systems  that  can  provide  full  six- 
degree-of-freedom  motion  at  the  target  tissue  despite  limited  access 
through  small  incisions  or  through  internal  body  pathways  such  as  the  GI 
tract,  the  bronchial  tree,  or  the  cardiovascular  tree. 

ii)  For  microsurgery,  provide  high  dexterity,  precise,  and  delicate  motion 
without  requiring  large  end  effectors  or  compromising  safety  or  sterility. 

iii)  For  image-guided  surgery,  develop  new  manipulators  that  can  work  pre¬ 
cisely  but  unobtrusively  with  a  variety  of  imaging  equipment  such  as  fluo¬ 
roscopic  C-arms  and  biplanar  devices,  CT  &  MRI  scanners,  and  3D 
ultrasound. 

iv)  Development  of  low-cost,  compact  systems  for  multiple  applications  that 
can  work  with  very  high  reliability  and  that  can  be  easily  sterilized  using 
commonly  available  means  such  as  autoclaving. 

c)  Substantially  enhance  the  “higher  level”  control  available  for  robotic  systems  in 
the  operating  room,  to  enable  them  to  function  more  as  surgical  assistants  rather 
than  only  as  teleoperated  slaves.  This  will  require: 

i)  Development  of  means  for  characterizing  common  surgical  tasks  in  terms 
of  models  of  patient  anatomy. 

ii)  Development  of  means  for  updating  these  models  from  intra-operative 
information  and  for  interpreting  surgeon  commands,  based  on  the  current 
surgical  context. 

iii)  Human-factors  research  to  determine  what  surgeons  actually  do  and  feel 
when  they  perform  various  procedures.  In  addition  to  providing  essential 
information  for  research  on  assistive  systems,  such  information  will  be 
generally  useful  in  establishing  performance  criteria  for  surgical  robots. 
For  example,  the  rapid  growth  of  laparoscopic  surgery  reveals  that  force 
information  and  full  six  degree-of-freedom  (DOF)  mobility  are  not  essen- 
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tial  for  some  procedures.  What  additional  procedures  are  possible  if  limited 
force  information  is  available?  With  high-fidelity  force  feedback?  With  an 
additional  DOF,  or  more  than  six  DOF? 

iv)  Integration  of  this  information  into  “higher  level”  controls  extending  tradi¬ 
tional  teleoperation  into  various  forms  of  supervised  autonomy.  One  partic¬ 
ular  challenge  for  such  systems  will  be  how  to  preserve  safety  and 
verification  of  surgeon  intentions  without  becoming  unduly  burdensome  on 
the  surgeon. 

3.  Grand  Challenges  -  The  robotics  task  force  spent  only  a  limited  amount  of  time  discuss¬ 
ing  possible  “grand  challenge”  applications  that  might  be  used  to  motivate  some  of  the 
above  research.  These  possibilities  represent  one  natural  evolution: 

a)  “Robotic  intern”  -  A  system  that  can  perform  many  of  the  same  manipulation  func¬ 
tions  now  performed  by  novice  surgical  personnel,  such  as  scope  pointing  and 
retraction.  Key  characteristics  and  research  challenges:  natural  interface  (speech 
recognition,  grab-and-move),  low  cost,  small  OR  footprint. 

b)  “Nostalgic  telesurgery  surgery”  -  A  system  for  minimally  invasive  surgery  that 
restores  to  the  surgeon  the  full  dexterity,  mobility,  and  sensitivity  enjoyed  in  open 
incision  procedures.  The  system  would  work  through  small  incisions  or  intralumi- 
nally,  but  the  surgeon  would  not  experience  constraints  of  present  endoscopic  or 
catheter-based  techniques.  Key  characteristics  and  research  challenges  include 
dexterous  and  compact  manipulators  (both  master  and  slave);  visual,  force  and  dis¬ 
tributed  tactile  feedback;  natural  user  interface;  and  safety. 

c)  “Robotic  resident”  -  A  system  that  can  perform  as  well  as  a  surgical  resident  in 
mid-training  in  performing  specific  tasks  under  the  supervision  of  an  experienced 
surgeon.  This  would  require  a  semi-autonomous  robot  which  incorporates  exten¬ 
sive  sensing  and  planning,  can  dissect  tissue,  suture,  etc.  Sensing  and  control 
issues  are  paramount  for  construction  of  such  a  system  and  include  how  to  deal 
with  soft  tissue,  sense  state  of  procedure,  re-plan  in  real  time,  interact  with  sur¬ 
geon,  safety,  etc. 

d)  “Super-delivery”  -  A  versatile  robotic  system,  integrated  with  a  configurable  vari¬ 
ety  of  intra-operative  and  pre-operative  imaging  modalities,  capable  of  navigating 
a  minimal  damage  path,  and  accurately  delivering  a  pattern  of  localized  therapy 
into  arbitrary  soft  tissue  lesions. 

4.  Evaluation  -  Better  means  for  evaluating  computer-assisted  surgical  techniques  must  be 
developed.  Many  current  clinical  outcome  measurement  tools  are  inappropriate  for  the 
accurate  evaluation  of  the  efficacy  of  computer-assisted  surgical  technology.  In  developing 
appropriate  measures,  active  participation  by  clinicians,  technology  researchers,  and 
human-factors  experts  are  all  essential. 

5.  Infrastructure  -  Support  development,  replication,  and  sharing  of  common  systems  infra¬ 
structure  and  component  subsystems  to  promote  research,  and  to  simplify  eventual  clinical 
qualification  and  deployment  of  the  results  of  research. 
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a)  As  in  many  areas  of  research,  medical  robotics  teams  often  must  spend  consider¬ 
able  effort  building  up  the  necessary  infrastructure  of  proven  components  (robots, 
sensors,  end-effectors,  systems  software,  registration  algorithms,  etc.).  Because  of 
safety  and  other  considerations,  the  delay  and  cost  associated  with  this  activity  are 
even  more  pronounced  than  in  other  areas  of  robotics  research  and  can  be  a  definite 
impediment  to  entry  into  the  field.  Some  specific  steps  that  might  be  taken  include 
creation  of  common  interfaces  and  exchange  standards,  hardware  and  software 
engineering  activity  to  facilitate  sharing  of  components,  and  encouragement  of 
“pooled”  equipment  acquisitions  and  shared  software  development  to  amortize 
costs  and  promote  interchange  of  results  and  techniques. 

b)  Similarly,  robotic  devices  have  significant  potential  to  enable  development  of 
novel  therapeutic  methods  that  can  significantly  improve  both  clinical  outcomes 
and  reduce  costs.  However,  such  synergistic  research  requires  a  more  rigorous 
level  of  engineering  than  is  commonly  found  in  typical  technology-oriented  aca¬ 
demic  research,  and  it  is  crucial  to  provide  a  bridge  between  initial  feasibility  pro¬ 
totypes  and  systems  that  are  effectively  usable  without  constant  attendance  by  their 
original  implementors. 

6.  Safety  -  Appropriate  safety  architectures  and  procedures  need  to  be  implemented  in  any 
medical  robotics  or  computer-assisted  surgical  system.  In  determining  what  is  appropriate, 
it  is  important  to  consider  the  potential  for  harm,  and  the  risks  inherent  in  conventional 
manual  procedures.  Further,  computer-controlled  medical  devices  themselves  raise  a  num¬ 
ber  of  important  research  issues,  as  discussed  in  Section  4.4.4,  that  should  be  pursued 
within  the  context  of  specific  applications  and  systems. 
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Chaired  by:  Scott  Delp  and  Ferenc  Jolesz 

5.1  Executive  Summary 

Surgical  Simulation  -  the  use  of  medical  imaging,  computer  graphics,  biomechanical  analysis, 
and  virtual  environments  to  simulate  surgery  for  medical  education,  scientific  analysis  and 
pre-treatment  planning. 

The  following  list  represents  the  major  proposed  directions  for  future  research  initiatives  in 
the  area  of  Surgical  Simulation: 

•  Incorporation  of  functional  models  into  existing  anatomical  models.  The  development  of 
physiologically-based  models  will  allow  us  to  create  more  realistic  and  useful  surgical  sim¬ 
ulations  in  which  the  functional  consequences  of  a  proposed  intervention  can  be  predicted, 
surgical  options  can  be  explored,  and  results  optimized.  This  new  development  will  have 
widespread  applications  in  medical  education,  scientific  research,  patient  care,  and  many 
non-surgical  applications. 

•  A  “grand-challenge”  for  surgical  simulation  is  to  create  a  virtual  human  body  that  allows 
one  to  learn  not  only  normal  and  abnormal  anatomy  and  physiology,  but  also  to  dissect  ana¬ 
tomical  structures,  simulate  medical  interventions,  and  predict  outcomes  for  a  wide  variety 
of  procedures  and  situations  (e.g.,  automobile  accident).  The  virtual  human  model  would 
serve  as  a  fantastic  virtual  laboratory  for  research  and  education,  decreasing  the  need  for 
animal  and  cadaver  experiments  in  medical  education  and  training. 

•  The  development  of  new  surgical  simulators  will  be  accelerated  by  capitalizing  on 
advances  in  related  fields,  such  as  medical  imaging,  computational  modeling,  and  virtual 
environments. 

5.2  Definitions 

We  have  defined  surgical  simulation  in  three  broad  areas  as  outlined  in  Figure  5-1.  The  first 
area,  training  systems,  may  be  comprised  of  software-based  simulations  exclusively,  or  a  com¬ 
bination  of  hardware  and  software  elements.  The  second  area  comprises  tools  for  scientific 
analysis.  This  includes  tools  to  design  procedures  and  implants,  to  predict  outcomes,  and  to 
assess  failure  and  success  of  an  intervention.  The  third  area  is  surgical  pre-planning  for  the 
purpose  of  performing  an  actual  procedure  with  the  assistance  of  robots,  trackers,  jigs,  endo¬ 
scopes  etc.  Thus,  surgical  simulation  is  a  first  step  in  most  image-guided  and  robot-assisted 
surgeries.  The  boundaries  between  these  three  areas  are  not  distinct.  For  example,  a  well- 
tested  training  system  may  be  used  to  plan  a  surgery  by  substituting  actual  patient  data  for  an 
idealized  computer  model  of  the  patient. 
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Figure  5-1:  Taxonomy  for  surgical  simulation 


Figure  5-2  shows  an  anatomical  and  biomechanical  model  of  a  human  lower  body  which  can 
be  used  to  help  explain  the  causes  of  movement  abnormalities  and  predict  the  functional  con¬ 
sequences  of  surgical  interventions.  This  type  of  functional  simulation  has  significant  poten¬ 
tial  for  application  in  the  area  of  surgical  simulation. 

5.3  Research  Directions  and  Review  of  Current  Technology 

Surgical  simulations  have  provided  demonstrable  benefits  in  several  areas.  For  example,  in 
craniofacial  reconstruction,  surgical  pre-operative  planning  decreases  operative  times,  pre¬ 
dicts  postoperative  geometry  and  improves  surgical  outcomes.  In  neurosurgical  applications, 
tumors  can  be  accurately  located  and  removed  without  damage  to  healthy  tissue.  However, 
current  applications  are  limited  to  simple  anatomical  models.  Creating  a  more  comprehensive 
anatomical/physiological  model  would  allow  evaluation  of  the  functional  consequences  of  a 
proposed  surgery,  and  serve  as  a  test-bed  for  a  wider  variety  of  applications.  For  example,  the 
post-operative  capacity  of  an  organ  system  could  be  estimated  after  tumor  removal.  Surgical 
simulators  may  also  replace  the  trial-and-error  process  of  training  and  accelerate  the  acquisi¬ 
tion  of  experience  by  clinicians. 

Figure  5-3  outlines  four  components  of  a  surgical  simulation  model  which  we  feel  would  be 
necessary  to  develop  the  full  potential  of  this  technology.  These  components  include  informa¬ 
tion  about  normal  anatomy,  physiology  and  pathology,  biomechanics  and  biology. 

Development  of  the  technology  necessary  to  construct  the  proposed  “grand-challenge”  virtual 
human  model  has  several  important  benefits.  Computer  graphics  models  of  the  body  provide 
an  effective  vehicle  for  communication  with  patients  so  that  they  can  understand  proposed 
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Figure  5-2;  A  simulator  incorporating  anatomic  and  biomechanic  models  which  can  be  used 
to  explore  the  functional  consequences  of  a  surgical  procedure 
(courtesy  of  Scott  Delp). 


Figure  5-3:  Components  of  a  surgical  simulation  model. 
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improved  visualization  during  minimally  invasive  surgery.  Simulation  also  inspires  innovation 
and  development  of  new  procedures.  In  the  future,  simulators  are  likely  to  be  used  to  evaluate 
and  certify  clinicians. 

5.4  Technical  and  Research  Issues 

This  section  summarizes  technical  problems  which  must  be  solved  in  order  to  develop  the 
next  generation  of  surgical  simulators. 

5.4.1  Development  of  Effective  Modeling  Tools 

•  3D  segmentation  -  There  is  a  need  for  more  complex  tissue  characterization  and  therefore  a 
more  complex  feature- space.  Image-based  segmentation  should  be  automated  and  based 
upon  data  from  multiple  sources.  Segmentation  can  be  improved  using  knowledge  bases 
derived  from  anatomical  and/or  functional  models. 

•  4D  modeling  -  Multidimensional  tissue  characterization  requires  multimodality  data  inte¬ 
gration  or  image  fusion.  Modeling  should  include  time-dependent  changes  for  the  investi¬ 
gation  of  long-term  effects,  such  as  remodeling  of  joints  or  tumor  growth. 

•  Scaling  -  Linear  and  non-linear  scaling  methods  are  necessary  to  utilize  the  models  in 
patient  care.  These  methods  are  used  for  customizing  models  for  an  individual  patient.  The 
translation  of  model  parameters  and  features  to  the  patient  will  require  the  development  of 
computational  methods  and  registration  techniques. 

•  Deformations  -  Predictable  and  unpredictable  deformations  in  anatomical  tissue  require  the 
development  of  various  methods.  Elastic  warping  is  necessary  to  match  models  with  a 
patient's  anatomy.  More  complex  deformations  may  require  the  utilization  of  biomechani¬ 
cal  properties.  It  is  important  to  model  short  and  long  term  alterations  in  morphology  in 
order  to  assess  consequences  and  outcomes  of  surgeries. 

•  Statistical  models  -  It  is  necessary  to  collect  normative  data  which  describe  various  param¬ 
eters  of  the  human  body.  This  should  include  growth  and  developmental  data  and  character¬ 
ization  of  the  aging  process.  Statistical  analysis  of  surgical  results  and  long  term  outcomes 
may  provide  an  important  component  of  optimizing  surgical  strategies  by  modeling. 

5.4.2  Model  Integration 

The  integration  of  various  models  and  model-components  (i.e.,  anatomical  and  functional) 
into  a  single  model  (i.e.,  virtual  human  body)  is  a  challenging  task  which  will  require  an  infra¬ 
structure  for  collaboration  of  a  large  interdisciplinary  team.  Integration  of  the  model  would  be 
facilitated  by  image  databases,  common  anatomical  modeling  software,  knowledge  of  tissue 
material  properties,  sharing  of  key  algorithms,  and  other  commonly  used  utilities.  A  web  page 
on  the  Internet  would  be  the  logical  place  to  compile  this  information,  along  with  a  list  of  col¬ 
laborators,  laboratories,  and  personnel  working  in  this  area. 
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Development  of  the  model  should  be  complemented  with  the  integration  of  the  execution 
model  (i.e.,  virtual  surgery)  to  obtain  the  final  result  of  simulation.  After  scaling  and  customi¬ 
zation,  this  information  can  be  used  for  planning  and  eventually  performing  the  modeled  pro¬ 
cedure  on  an  individual  patient.  While  training  simulators  can  use  generic  models  of  the  body, 
patient- specific  models  are  required  for  this  type  of  surgical  planning. 

5.4.3  Enabling  Technologies 

Inexpensive,  robust  devices  for  interaction  with  surgical  simulators  are  needed.  For  example, 
haptic  displays,  specifically  created  for  medical  applications,  need  to  be  developed.  Stereo 
visualization  systems,  on  the  other  hand,  will  probably  be  developed  for  entertainment,  scien¬ 
tific  visualization,  and  other  applications  and  might  be  used  in  medical  applications.  In  addi¬ 
tion,  computationally  efficient  methods  for  simulating  tissue  deformation,  bleeding,  cutting 
and  tearing  are  needed. 

Computer  models  of  the  human  body  developed  for  entertainment  applications  are  not  appro¬ 
priate  for  use  in  scientific  analysis  and  patient  care.  While  simulations  developed  for  enter¬ 
tainment  or  demonstrations  at  trade  shows  may  appear  to  be  realistic  at  a  superficial  level,  they 
frequently  do  not  account  for  the  underlying  mechanics  and  physiology.  This  occurs  because 
of  the  fundamental  differences  between  simulations  developed  for  medicine  and  those  devel¬ 
oped  for  entertainment  (Table  5-1). 


Table  5-1:  Differences  between  computer  graphics  in  medicine  and  entertainment 


Medical  Graphics 

Graphics  for  entertainment 

•  Physically  realistic 

•  Visually  (sensually)  realistic 

•  Intolerant  of  artistic  license 

•  Appreciate  and  encourage  artistic 
enhancement 

•  Interactivity  needs  flexibility  and 

adaptability 

•  Limited  paths  (i.e.,  pre-computed 

•  Truth  is  of  absolute  importance 

actions/scenes)  acceptable 

•  Good  story  is  the  most  important  com¬ 
ponent  (i.e.,  fiction) 

Simulations  must  be  tested  extensively,  both  in  terms  of  their  accuracy  and  their  efficacy.  This 
involves  collection  of  basic  data,  tests  at  each  level  of  simulation,  and  comparison  with  clini¬ 
cal  results. 

5.5  Summary  /  Recommendations 

Our  group  focused  on  the  coupling  of  functional  and  anatomical  models.  The  development  of 
physiologically-based  models  will  allow  the  creation  of  more  realistic  and  useful  surgical  sim¬ 
ulations  in  which  the  functional  consequences  of  a  proposed  intervention  can  be  predicted, 
surgical  options  can  be  explored,  and  results  optimized.  This  new  development  will  have 
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widespread  applications  in  surgery,  medical  education,  scientific  research,  patient  care,  and 
non- surgical  applications.  Example  applications  of  simulation  in  surgery  are  outlined  in 
Table  5-2. 


Table  5-2:  Potential  Application  Areas  of  Surgical  Simulation. 


Neurosurgery 

•  Functional  neurosurgery 

Musculoskeletal  Surgery 

•  Limb  reconstruction 

•  Joint 

•  Spine 

•  Trauma 

•  Tumor 

•  All 

Plastic  Surgery 

•  From  skeletal  structures 

*  Facial  simulations 

*  Breast 

*  Donor  defects  bums 

Minimallv-Invasive  Surgery 

•  Endoscopy 

•  Bronchoscopy 

•  ENT  -  sinus 

•  Endoyascular 

Soft-tissue  Surgery 

•  Reconstruction  of  tumor 
yolume  and  calculation 
of  residual  functional 
capacity  for  diseases 
such  as  liyer  cancer,  lung 
cancer  and  respiratory 
failure 
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Section  6  -  Teleinterventions 

Chaired  by:  Jon  Bowersox  and  Dietrich  Gronemeyer 

6.1  Executive  Summary 

Teleintervention  -  the  application  of  information-based  technologies  to  deliver  procedural 
health  care  through  an  electronic  interface.  Indirect  patient  contact  is  implicit;  however,  the 
distance  separating  patient  and  physician  may  be  insignificant,  or  may  be  great. 

The  following  list  represents  the  major  proposed  directions  for  future  research  initiatives  in 
the  area  of  Teleinterventions: 

•  The  highest  priority  technical  needs  are  in  user  interface  optimization,  system  validation, 
haptic  tools,  and  the  development  of  redundant  controllers.  Emphasis  should  also  be  placed 
on  establishing  early  relationships  with  regulatory  agencies  and  national  health  systems  to 
ensure  the  timely  and  appropriate  introduction  of  complex  new  technologies. 

•  The  greatest  user  need  and  value  for  teleintervention  will  be  in  systems  that  are  used  locally 
to  enhance  dexterity,  and  reduce  the  risks,  time,  and  costs  associated  with  complex  proce¬ 
dures. 

6.2  Introduction,  Definitions  and  Description  of  Area 

The  development  of  information-based  technologies  has  enabled  physicians  to  perform  com¬ 
plex  therapeutic  procedures  on  patients  without  directly  touching  them.  Laparoscopic  surgery 
is  the  most  widely  applied  example  of  this  concept.  Surgeons  indirectly  view  intra-abdominal 
organs  through  video  display  interfaces,  and  tissue  manipulation  is  performed  indirectly 
through  long,  thin  instruments  inserted  through  5  mm  portals.  Although  highly  successful,  the 
performance  limitations  imposed  by  minimizing  tissue  access  have  created  the  need  for  sys¬ 
tems  that  restore  natural  function  and  dexterity  to  surgeons.  Furthermore,  the  realization  that 
therapy  could  be  delivered  through  indirect  visualization  and  manipulation  of  tissues  stimu¬ 
lated  interest  in  applying  computer-assisted  medical  interventions  to  locations  that  were  previ¬ 
ously  inaccessible  because  of  size,  minimal  entry  apertures,  distance,  or  hazardous 
environments. 

Teleintervention  is  a  new  term  used  to  describe  the  application  of  information-based  technolo¬ 
gies  to  deliver  procedural-based  health  care  through  an  electronic  interface.  Teleintervention 
is  distinct  from  image-guided  therapy  in  that  feedback  to  and  from  the  clinician  is  via  an  elec¬ 
tronic  interface.  In  contrast  to  robotics,  direct,  human-in-the-loop,  operator  control  of  manipu¬ 
lations  occurs.  Teleintervention,  however,  may  use  techniques  from  image-guided  therapy, 
medical  robotics,  and  surgical  simulators  as  system  components.  Teleintervention  encom- 
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passes  the  fields  of  telepresence  surgery,  telemanipulation,  and  teleoperation,  in  which  an 
operator’s  hands  manipulate  remote  tissues.  It  also  describes  remote  teleconsulting,  telemen¬ 
toring,  and  teleproctoring,  in  which  procedures  are  observed  and  guided  from  remote  loca¬ 
tions  (e.g.,  remote  control  of  cameras,  monitoring  instruments,  and  ancillary  devices),  but  in 
which  no  actual  manipulation  of  remote  tissues  is  performed. 

Teleintervention,  as  defined  here,  does  not  include  video  teleconferencing  for  distributed  med¬ 
ical  education,  pre-operative  diagnosis,  or  post-procedural  care.  Nor  does  it  include  current 
methods  of  minimally  invasive  surgery  in  which  an  operator’s  hands  contact  a  patient’s  tissues 
through  a  mechanical  linkage  (i.e.,  a  surgical  instrument).  It  also  does  not  include  telemedi¬ 
cine  applications  in  non-procedural  based  care,  such  as  diagnostic  radiology,  cardiology,  der¬ 
matologic  diagnosis,  or  psychiatry.  In  teleintervention,  the  distance  from  the  provider  to  the 
patient  may  be  less  than  a  meter  when  using  a  micromanipulator  system,  or  as  great  as  several 
thousand  kilometers  (or  more)  when  providing  teleconsultation  to  remotely  located  operating 
rooms. 

Figure  6-1  shows  a  tele-operated  master-slave  manipulator  with  bi-directional  force  feedback 
which  has  been  developed  for  surgical  applications.  This  system  has  been  validated  in  a  num¬ 
ber  of  animal  studies. 


Figure  6-1:  Remotely  operated  master  device  (left)  controlling  a  tele-operated  slave 
manipulator  (right)  in  a  simulated  surgical  environment  (courtesy  of  Jon 
Bowersox). 
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6.3  Review  of  Current  Technology 

A  number  of  teleinterventional  systems  in  development  are  listed  in  Table  6-1.  Active 
teleintervention  has  been  successfully  demonstrated  using  the  SRI  Telepresence  Surgery  Sys¬ 
tem  in  a  variety  of  pre-clinical  studies.  Professor  Angelini  in  Rome  and  Professor  Wells  in  the 
UK  have  used  a  precursor  of  the  MIDSTEP  system  to  demonstrate  the  feasibility  of  remote 
manipulation  for  a  liver  biopsy,  and  will  soon  begin  a  multi-institutional  study;  however,  to 
our  knowledge  there  are  no  surgical  telemanipulator  systems  currently  being  used  in  clinical 
trials.  Passive  and  assistive  surgical  teleintervention  have  been  performed  on  patients  at  Johns 
Hopkins  University,  as  described  in  several  recent  peer-reviewed  publications.  Informal  clini¬ 
cal  studies  have  been  initiated  by  the  U.S.  Department  of  Defense  between  military  surgeons 
in  Bosnia  and  military  medical  centers  in  the  U.S.  Other  active,  passive  and  assistive  teleinter¬ 
ventional  systems  are  in  various  phases  of  preliminary  development  in  the  U.S.,  Europe,  and 
Japan. 

The  configuration  of  teleintervention  systems  varies  based  on  anticipated  needs,  but  compo¬ 
nents  common  to  all  include  a  physician,  a  patient,  and  an  electronic  interface  linking  the 
patient  with  the  treatment  provider  (see  Figure  6-2).  Systems  may  include  robotic  manipula¬ 
tors,  imaging  systems  and  overlays,  and  networking/telecommunications  components. 


f JL  JGiXjCi  Jt JKjEb''il3lJEbJL^ 
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Figure  6-2:  Prototypical  teleintervention  system  (courtesy  of  Jon  Bowersox). 


Systems  currently  in  development  include  video  and  audio  displays,  haptic  interfaces,  com¬ 
puters  (e.g.,  controllers,  DSP,  encryption),  telecommunications  interfaces,  and  ancillary  com¬ 
ponents  (e.g.,  monitoring  devices,  electrocautery  and  suction  controls).  Future  systems  will 
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also  incorporate  image  overlay  technology  from  local  and  networked  sources,  simulation 
capabilities,  and  multi-user  environments. 

6.4  Technology  and  Research  Issues 

6.4.1  Relationship  to  other  RCAMI  Application  Areas  and  Enabling  Technologies 

All  of  the  enabling  technologies  identified  in  the  pre-workshop  report  (see  Appendix  I)  have 
potential  application  benefits  for  teleintervention;  however,  none  of  the  current  teleinterven¬ 
tion  systems  evaluated  is  dependent  on  these  technologies  for  implementation.  One  advantage 
of  surgical  teleoperators  over  technologies  currently  available  for  image-guided  therapy  is  the 
ability  to  precisely  manipulate  soft  tissues.  A  limitation,  however,  is  the  reliance  on  video  as 
the  sole  source  of  information  on  tissue  characteristies.  Dynamic  overlays  of  digital  imaging 
data  (e.g.,  MRI,  CT)  may  enable  a  greater  range  of  teleintervention  applications  than  is  cur¬ 
rently  envisioned.  Furthermore,  sensory  enhancement  by  fusing  multiple  sensing  sources  may 
compensate  for  sensory  limitations,  and  allow  more  intuitive  operation  of  complex  systems 
(i.e.,  telepresence). 

Combining  teleintervention  systems  with  surgical  simulators  may  enhance  the  capability  to 
train  for  complex,  or  infrequently  performed  procedures,  and  may  allow  pre-planning,  with 
real-time  review  during  an  operative  procedure.  Likewise,  incorporating  robots  with  autono¬ 
mous  or  supervisory  level  control  into  teleintervention  systems  may  enable  operators  to  gain 
additional  dexterity,  deerease  time  and  ancillary  labor  requirements,  and  facilitate  multitask¬ 
ing. 


6.4.2  Technical  Limitations  and  Needs 

Several  fundamental  technical  problems  and  research  needs  that  are  key  to  further  develop¬ 
ment  of  teleintervention  systems  were  identified. 

•  Human  Computer  Interface 

-  Ergonomics  (instrument  design)  -  available  instrumentation  has  generally  been 
adapted  from  traditional  instruments  and  other  electronic  systems,  without  particular 
regard  to  cost  or  productivity  efficiency.  Telemanipulators  require  the  development 
of  optimized  instrumentation  at  the  master  and  slave  locations.  Furthermore,  the 
focus  on  microinvasive  procedures  has  created  a  requirement  for  novel  instruments 
that  can  be  used  in  conjunction  with  endoscopic  and  image-guided  techniques.  There 
is  a  need  for  MRI-compatible  surgical  manipulators  and  instruments. 

-  Visual  Display  (orientation,  dynamic  variable  resolution,  stereopsis,  frame  rate, 
head-mounted  display)  -  the  quality  and  information  requirements  for  teleinterven¬ 
tion  applications  need  to  be  defined,  with  the  goal  of  optimizing  data  needs. 
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-  Audio  Input  -  spatial  and  qualitative  presentation  of  audio  input  need  to  be  more 
fully  developed  for  specific  applications. 

-  Human  Performance  Engineering  -  the  need  exists  to  define  critical  cognitive,  per¬ 
ceptual,  and  motor  tasks  involved  in  procedural  health  care,  and  to  develop  qualita¬ 
tive  and  quantitative  performance  measures  that  can  be  used  to  benchmark 
application-specific  component  requirements.  Furthermore,  performance  measures 
need  to  be  developed  and  validated  for  comparing  teleintervention  systems  with  each 
other,  and  with  existing  models  of  health  care  delivery. 

•  Communications/Networking 

-  Signal  Quality 

-  Latency 

-  Signal  Processing  -  including  compression  schemes 

-  Bandwidth  Requirements 

-  Traffic  Prioritization 

-  Controllers 

-  Security  (encryption) 

-  Network  Architecture  (scaleability,  reliability,  redundancy) 

-  Integration  with  Hospital  Information  Systems  (electronic  patient  records,  billing, 
scheduling) 

•  Imaging 

-  Sensory  amplification  (augmented  reality)  -  includes  synthetic  data  sets  from  simula¬ 
tors,  as  well  as  cue  data  to  define  exclusion  regions  (anatomic  danger  zones). 

-  Image  overlays  -  the  incorporation  of  image  registration,  and  overlay  of  digital  ana¬ 
tomic  and  physiologic  data  sets  (e.g.,  duplex  ultrasound  flow  data  and  images)  need 
to  be  developed. 

•  Haptic/Tactile  Devices 

-  Haptic  Interface  (tactile  sensing,  force  amplification,  latency,  fidelity,  control  sys¬ 
tems,  DOF)  -  as  a  key  interface  feature  of  telemanipulators,  there  is  a  critical  need 
for  both  tactile  and  kinesthetic  informational  systems. 

•  System  validation  (performance  measures) 

-  Internal  (technical  evaluation  of  components) 

-  External  (application-specific  system  performance  measures) 

•  Control  Systems 

-  Redundancy  -  software  and  hardware  systems  for  avoiding  loss  of  linkage  with 
remote  site,  ensuring  adequate  SNR,  emergency  protocols  (e.g.,  robotic  safety) 

-  Real-time  Data  Integration  -  from  multiplexed  sensor  sources,  databases,  system 
components 
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•  Safety/Standardization 

-  Tolerance  Limits  for  Tissue  Forces 

-  Cross-Industry  Analysis  of  Safety  Standards/Tolerances  for  Automated  and  Robotic 
Systems  (e.g.,  aviation  industry) 

-  Sterilization  of  highly  complex  and  miniaturized  manipulators  and  system  compo¬ 
nents 

-  International  specification  standards  to  ensure  interconnectibility 

•  Microsensor/Actuator  Development 

-  Component  miniaturization  to  incorporate  into  minimal  access  interventional  sys¬ 
tems  (minimally  invasive  surgery,  intraluminal  and  cavitary  endoscopy,  catheter- 
based  therapeutic  systems) 

6.5  Existing  Deficiencies  and  Problems 

A  working  relationship  needs  to  be  developed  between  systems  developers  (academic  and 
industrial)  and  the  FDA.  The  complexity  and  novelty  of  teleintervention  solutions  to  clinical 
problems,  and  the  potential  reliance  on  robotic  and  computer  systems  for  therapy  necessitate 
heightened  awareness  on  the  part  of  both  developers  and  regulators.  This  is  distinct  from  any 
proposal  to  form  high-level,  multi-agency  task  forces,  which  may  have  some  value  in  gaining 
end  user  awareness  and  acceptance.  A  relationship  with  the  U.S.  Health  Care  Financing 
Administration  (HCFA)  and  major  health  care  purchasers  (e.g.,  Columbia-HCA)  should  be 
established  to  define  buyer  needs,  limitations,  and  expectations. 

Validation  is  critical  for  the  development,  justification,  and  clinical  acceptance  of  teleinterven¬ 
tion  systems,  yet  no  adequate  measures  of  complex  medical  task  performance  have  been 
developed.  This  is  a  reflection  of  the  high  skill  level  required  for  many  procedures  in  the  surgi¬ 
cal  specialties,  and  the  multidimensional  contributions  of  cognitive,  perceptual,  and  psycho¬ 
motor  expertise.  It  will  be  difficult  to  develop  systems,  demonstrate  safety,  and  prove  efficacy. 
Outcomes  data  will  be  useful,  but  the  low  morbidity  and  mortality  of  current  therapeutic 
modalities,  and  the  long  duration  of  well-designed  outcome  studies  will  likely  require 
extremely  large,  multicenter  clinical  trials  to  achieve  sufficient  statistical  power.  Furthermore, 
the  short  product  life  cycle  of  technological  innovations  will  jeopardize  the  value  of  such  stud¬ 
ies. 

The  need  exists  to  perform  research  in  developing  validation  measures  that  will  be  quantita¬ 
tive,  reproducible  among  subjects  and  systems,  and  that  can  be  tailored  to  specific  applica¬ 
tions.  Component  (engineering)  validation  will  be  needed  for  intra-system  validation, 
including  system  integration,  networking,  control  issues,  and  technical  specifications.  Systems 
validation  will  be  needed  to  assess  effectiveness  in  performing  clinical  tasks.  It  is  unlikely  that 


45 


Section  6  -  Teleinterventions 


broad  application  of  single  measures  will  result  in  adequate  sensitivity  for  assessing  all 
requirements. 

Patient  and  operator  safety  were  repeatedly  stressed  as  key  concerns  that  would  hinder  system 
acceptance  by  users  (physicians  and  patients),  and  by  regulatory  agencies.  Although  commer¬ 
cial  development  will  provide  some  of  the  impetus  for  redundant  control  systems,  basic  tech¬ 
nical  issues  that  need  to  be  addressed  include  defining  tolerable  force  limits  on  human  tissues, 
developing  software  to  ensure  redundancy  for  local  manipulator  systems,  and  data  integrity, 
security,  and  redundancy  for  networked  transmissions.  Sterilization  issues  (requirements, 
techniques)  for  complex,  reusable  manipulators  and  instrumentation  also  need  to  be  defined. 
Other  existing  deficiencies  are  in  international  standards  for  teleintervention  systems  specifi¬ 
cations  and  description.  Non-technical  issues  raised  include  medicolegal  liability,  patient 
acceptance,  and  the  ethics  of  health  care  delivery  using  an  electronic  interface. 

6.6  Summary  /  Recommendations 

The  benefits  of  local  teleintervention  systems  will  be  enabling  completely  new  procedures  that 
cannot  be  performed  now  because  of  limitations  in  dexterity,  access  (minimally  invasive, 
microscopic,  intraluminal  endoscopy),  or  complexity.  Intra-hospital  systems  will  also  be  used 
for  skill  acquisition  and  retention  (training),  using  a  simulated  environment.  These  systems 
are  likely  to  be  implemented  in  low  volume,  at  tertiary  and  quaternary  medical  centers. 

Networked,  or  remote,  teleintervention  systems  will  find  the  greatest  use  in  primary  care, 
community  settings.  They  will  be  used  to  increase  patient  access  to  procedures  performed 
locally  (e.g.,  enabling  general  surgeons  to  perform  standard  urologic  or  otolaryngologic  pro¬ 
cedures  under  the  mentoring  of  a  remotely  located  specialist),  optimize  the  distribution  of 
health  care  resources,  and  enhance  training. 

An  additional  application  area  meriting  evaluation  is  in  delivering  anesthesia  care  remotely. 
Anesthesia  providers  currently  interact  with  patients  through  electronic  and  mechanical  inter¬ 
faces,  thus  making  the  transition  to  teleanesthesia  care  relatively  easy.  The  potential  to  provide 
closer  supervision  of  nurse  anesthetists  and  physicians  in  graduate  medical  education,  and  the 
ability  to  observe  and  mentor  less  experienced  anesthesia  providers  through  rare  or  complex 
cases  performed  at  a  community  hospital  are  direct  potential  benefits  of  teleinterventional 
technology.  Directly  achievable  cost  savings  and  improved  patient  outcomes  are  possible 
through  this  application. 
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Workshop  participant  breakdown; 

Table  A- 1:  Workshop  Statistics  (Participants  Only) 


Participating  Countries 

M.D. 

Ph.D. 

Other 

Total 

United  States 

8 

10 

2 

20 

Germany 

2 

2 

4 

United  Kingdom 

6 

8 

14 

Japan 

5 

3 

8 

France 

3 

3 

Switzerland 

1 

1 

Italy 

1 

1 

2 

Total 

22 

28 

2 

52 
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David  A.  Simon 
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Keynote: 
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Robotics/Manipulators 

Co-Chairs: 

David  Stulberg,  M.D. 

jointsurg  @  nwu.  edu 
Russell  Taylor,  Ph.D. 
rht@cs.jhu.edu 

Keynote: 

Russell  Taylor,  Ph.D. 
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Participants: 
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Keynote: 
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Appendix  C  Robotics  Keynote 

Keynote  Address  given  by:  Russell  H.  Taylor 


Medical  Robotics 
and  Computer-Assisted  Surgery 

Robot  ^  Surgeon 

Russell  H.  Taylor 

Department  of  Computer  Science 

The  Johns  Hopkins  University 

So  what  is  “Medical  Robotics”? 

Application  of  “robotic”  technologies 
-Sensing 
-Manipulation 

-Modelling  and  Geometric  Analysis 
-Human-Machine  Interfaces 
to  enhance  human  clinicians’  ability  to  plan 
and  carry  out  interventional  procedures 

Robot  =  Surgical  Tool 

Robot  +  Human  >  Unaided  Human 


A  plan  must  be  executable 
if  it  is  to  be  useful 
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Taxonomies 

•  By  Technology/Research  Area 

•  By  Application 

•  By  Class  of  System 


Technology/Research  Area 

•  Image  Processing 

•  Analysis 

-  Acquisition 

-  Modelling 

-  Segmentation 

-  Simulation 

-  Registration 

-  Optimization 

-  Visualization 

-  Visualization 

Technology/Research  Area 

•  Robotics 

•  Systems 

-  Sensing 

-  Safety 

-  Manipulation 

-  Robustness 

-  Human-machine 

-  Information 

interfaces,  including 

management  and 

visualization 

communication, 
including  visualization 

Application  Areas 

•  Neurosurgery 

•  Dentistry 

•  Orthopaedics 

•  Microvascular 

•  Opthomology 

•  Oncology 

•  General  Surgery 

•  Podiatry 

•  Urology 

•  Otolaryngology 

•  Craniofacial 

•  etc. 

Considerations 

Factors  to  consider 

Points  of  leverage 

•  Safety 

•  New  Procedures 

•  Efficacy 

•  Consistency 

•  Utility 

•  Time 

•  Market 

•  Cost 

•  System  cost 

•  Complication  rates 

•  Regulatory  Issues 

•  Healing  times 

•  Enabler 

Points  of  Leverage 

How  can  the  inequality  be  satisfied? 


Ability  to  do  it  at  all 

Consistency 

Time 

Cost 


•  Safety  to  clinicians 

•  Safety  to  patients 

•  Healing  times 

•  etc. 
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The  Roles  Systems  Play 

Navigational  Aids 

•  Navigational  Aids 

•  ‘Intern  Replacements” 

•  Telesurgical  Systems 

•  Precise  Positioning  Systems 

•  Precise  Path  Systems 

•  Goal  is  positional  information  support 

•  Often  called  “CAS”  systems 

•  Usually  consist  of 

-  localizer 

-  registration 

-  display 

“Intern  Replacements” 

Telesurgical  Systems 

•  Retractors 

•  Surgeon  controls  “master” 

•  Instrument  Clamps 

•  “Slave”  robot  follows  motions 

•  Limb  Positioning  Systems 

•  But ...  can  be  made  much  smarter 

•  Laparoscopic  Camera  Holding  Systems 

•  Local  Assistive  Surgery 

•  Remote  Surgery 

Remote  Surgery 

Local  Assistive  Surgery 

Operate  at  a  distance 

Use  system  to  extend  surgeon’s  capability 

-  Military/disaster  applications 

-  Laparoscopic  surgery 

-  Remote  health  care  delivery 

-  Eye/microsurgery 

Main  Challenges: 

-  Time  Delay 

Main  Challenges: 

-  “Presence” 

-  Surgeon  interface 

-  Functions 
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Steady  Hand  Manipulation 

Precise  Positioning  Systems 

•  Hands  on  guiding  with 

-  actively  enforced  constraints 

-  shared  autonomy 

•  Typical  applications: 

-  Orthopaedics 

-  Neurosurgery 

-  Microsurgery 

•  E.g,,  Davies,  Taylor, ... 

•  Goal  is  accurate  positional  alignment 

•  Provide  manipulation  assistance 

-Passive  -  e.g.,  osteotomies 

-  Semi-active  -  e.g.,  point  biopsies 

-  Active  -  e.g,,  pattern  therapy 

Localized  Therapy  Delivery 

Localized  Therapy  Delivery 

Goal: 

Technical  challenges: 

•  Optimized  planning  of  therapy 

•  Proof  of  modality 

pattern  using  3D  images 

•  Image  Segmentation 

•  Robotically  assisted  delivery  using 

•  Image/robot  registration 

“light-weight”  intraoperative 
images 

•  Patient/organ  motion  &  deformation 

LIVER  CANCER 

Goal:  Percutaneous  Treatment 

•  Hepatomas 

•  Significantly  increase  indication  rate 

-  10,000  cases/year,  25%  operable 

-  Condition  of  patient 

•  Metastatic  Colon  Cancer 

-  Relative  benefit 

-  25,000  cases/year,  30%  operable 

•  Significantly  decrease  cost 

•  Average  surgery  charges  at  JHU  Hospital 

-  Perhaps  $6, 000-$7 ,000/case 

-  $22,000/case,  10  day  hospital  stay 

•  Significantly  reduce  hospital  stay 
-  Perhaps  1  -3  days 
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Percutaneous  Kidney  Puncture 

•  Goal  is  to  place  needle  into  the  filling 
system  of  the  kidney 

•  Initial  experiments: 

-  Biplanar  guidance,  LARS  robot 

•  Eventual  system 

-  Single  C-Arm,  New  Robot 


Precise  Path  Systems 

•  Goal  is  accurate  following  of  predefined 
path. 

•  E.g., 

-  Orthopaedics 

-  Radiation  beam  therapy 

•  Systems  can  be  both  active  and  passive 


Robodoc 

•  TBM  Research  +  UC  Davis 

-  Exploratory  work  (1986-1988) 

-  Prototype  system  (1989-1990) 

-  Veterinary  trial  (1990) 

•  Integrated  Surgical  Systems 

-  Phase  1  trial  (1 1/92-3/93) 

-  Phase  2  trial  (10/93-...) 

-  Commercial  deployment  in  Europe  (1995  -...) 


Hap’s  four  laws  of  robotics 

•  The  surgeon  is  the  boss 

•  Never  run  amok 

•  Never  push  too  hard 

•  Stay  where  you  are  supposed  to  be 


Insertability  Analysis 

Revision  Hip  Surgery 

•  Goal  is  determining  whether  implant  can  be 
inserted  into  prepared  hole. 

•  Approach 

-  Simulate  6  DOF  extraction  &  reverse 

Joint  Activity: 

•  Integrated  Surgical  Systems 

•  IBM  Research 

•  Johns  Hopkins  University 

-  Input  =  CAD  files 

-  Output  =  Path  or  “stuck”  configuration 

Joskowicz  &  Taylor  (1994,  1995) 
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Demographics 

•  Increasingly  common 

-23,000/year  (US,  1992) 

-  10%  annual  growth 

•  Expensive 

-  Average  cost  =  $24,000,  1 1  day  hospital  stay 

•  Error  Prone 

-  18%  fractures 

-  10%  cortical  penetration 


Revision  THR  Issues 

•  Planning 

-  CT  Reconstruction  artifacts 

-Use  of  (imperfectly  registered)  x-rays 

-  Segmentation 
-Contingency  planning 


Revision  THR  Issues 

•  Execution 

-  Registration 

-  Intraoperative  plan  verification 

-  Intraoperative  plan  modification 

-  Human-machine  interaction 


Wither  are  we  tending? 

•  “Medical  Robots”  are  still  surgical  tools 
intended  to  be  used  as  part  of  surgical 
systems. 

•  Systems  that  exploit  integration  of  images, 
models,  plans,  and  manipulation  will 
become  more  prevelant. 

•  Systems  will  be  both  subjects  of  research 
and  enablers  for  research. 


How  to  get  there 

•  Pick  good  problems 

-  Clinical  importance  and  technical  interest 

•  Invest  heavily  in  team  building 

-  Clinicians  and  Engineers 

•  Rapid  iteration  with  careful  evaluation 

•  Sharing  of  Lessons 
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Appendix  D  Surgical  Simulator  and 
Virtual  Reality  Modeling  in 
Orthopaedics 

Keynote  Address  given  by:  Edmund  Y.S.  Chao 

D.l  Introduction 

I  truly  appreciate  the  honor  of  being  invited  by  this  very  distinguished  group  to  discuss  one  of 
the  major  topics  of  this  workshop,  the  “Surgical  Simulator”.  With  the  amount  of  time  given 
to  me,  I  intend  to  comment  on  other  sub-topics,  hoping  to  set  the  stage  and  provide  examples 
for  our  deliberations  during  the  next  2  1/2  days.  Several  important  issues,  both  technical  and 
non-technical,  will  be  discussed.  I  would  like  to  add  to  the  list  provided  by  the  workshop 
chairman  relating  to  the  recognition  of  new  engineering  technology  transfer  to  medicine  and 
surgery,  and  hopefully,  to  share  the  spotlight  and  funding  currently  occupied  by  the  dominat¬ 
ing  fields  of  molecular  biology  and  gene  manipulation.  It  is  interesting  to  note  that  the  appli¬ 
cation  of  biology  and  physiology  principles  to  the  vast  field  of  engineering  technology,  under 
the  discipline  described  as  “Bionics”,  has  long  been  recognized  as  the  impetus  for  advance¬ 
ments  in  our  engineering  sciences  and  industry.  Therefore,  bioengineering  provides  a  two-way 
street  where  technology  cross-feeds  both  the  medical  and  the  engineering  fields.  However,  it  is 
important  to  emphasize  the  distinct  difference  between  the  new  technologies  proposed  for 
medical  education,  research,  and  services  (surgical  simulation,  virtual  reality  modeling, 
image-guided  procedures,  etc.)  and  those  used  in  multi-media  applications,  entertainment, 
advertising,  and  courtroom  deliberation. 

D.2  The  Scope  of  Surgical  Simulator  in  Orthopaedics 

This  workshop  deals  with  four  topics:  1)  Surgical  Simulators;  2)  Image-Guided  Procedures;  3) 
Robotics/Manipulators;  and  4)  Tele-Intervention.  The  surgical  simulator  relies  upon  Virtual 
Reality  models  and  physical  models  which  can  be  used  for  pre-treatment  planning,  medical 
research  and  education.  Supplementary  to  its  clinical  applications.  Virtual  Reality  models  can 
yield  biomechanical  information  unavailable  from  in  vivo  or  in  vitro  studies.  Realistic  physio¬ 
logical  movement  and  joint  reaction  forces  in  the  musculoskeletal  systems  can  be  determined 
using  Virtual  Reality  biomechanical  models  and  validated  through  mechanical  testing  on 
cadaver  specimens. 

The  motivation  for  developing  the  concept  of  a  surgical  simulator  in  orthopaedic  surgery  was 
stemmed  from  the  treatment  of  musculoskeletal  neoplasm.  When  treating  a  bone  or  soft  tissue 
tumor,  a  special  PET  scan  is  often  used  to  characterize  and  localize  the  lesion.  After  pin-point- 
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ing  the  precise  margin  of  the  lesion,  the  surgeons  can  use  computer-guided  biopsy  and  treat 
the  affected  region  with  imaging-guided  procedures  such  as  focalized  radio-therapy  for  tumor 
control.  Following  medical  management,  surgical  resection  is  often  indicated.  When  perform¬ 
ing  a  resection,  the  boundary  of  the  lesion  must  be  safely  contained  by  a  layer  of  normal  tis¬ 
sue,  thus  necessitating  removal  of  a  large  volume  of  bone  and  soft  tissue,  followed  by  major 
reconstruction.  After  such  a  radical  resection,  surgical  reconstruction  can  be  planned  using 
computer-aided  implant  design  and  image-guided  implantation.  Finally,  optimal  and  safe 
rehabilitation  program  can  be  planned  using  a  Virtual  Reality  model  of  the  reconstructed 
region  in  order  to  regain  functional  use  of  the  salvaged  limb  and  joint  system.  Such  an  inte¬ 
grated  application  of  the  new  engineering  technology  helps  to  set  the  stage  for  the  present  dis¬ 
cussion  on  surgical  simulation  and  the  use  of  virtual  reality  modeling  in  orthopaedics.  It  also 
exemplifies  the  value  of  such  technology  as  applied  to  medical  education,  surgical  training, 
basic  research  and  clinical  patient  care  covering  all  aspects  of  orthopaedics. 

D.3  Role  of  VR  Modeling  (Virtual  Human)  in  Surgical  Simulation 

The  concept  of  robotics  and  its  medical  application  played  an  important  role  in  the  present 
workshop  and  it  has  specific  implication  to  the  development  of  surgical  simulation.  In  this 
respect,  the  entertainment  world  shared  the  initial  stimulation.  The  underlying  plot  of  the 
movie,  Jurassic  Park,  was  medical  and  biological  based,  an  interesting  application  of  the 
chaotic  theory  in  genetic  mutation.  More  importantly,  it  was  the  remarkable  Virtual  Reality 
modeling  and  mechanical  simulation  of  the  dinosaurs  that  gave  a  lifelike  effect  through  com¬ 
puter  graphics  and  robotic  manipulation.  Such  computer  and  engineering  marvels  provided 
the  impetus  to  the  development  of  the  Virtual  Human  model.  The  virtual  human  is  a  pro¬ 
grammable  human  musculoskeletal  system  on  a  computer  workstation  that  can  give  us  the 
dynamic  responses  of  the  system  under  applied  loads  and  measured  kinematics.  The  associ¬ 
ated  biomechanical  information  related  to  muscle  forces,  stresses  in  the  bone  and  joint  under 
simulated  physiological  function  not  only  can  be  quantitated  but  also  displayed  in  an  animated 
fashion  together  with  the  graphic  model.  Surgical  simulation  as  well  as  the  remaining  topics 
of  this  workshop  will  require  the  virtual  human  model  to  facilitate  their  utility. 

What  are  the  advantages  that  a  surgical  simulator  utilizing  virtual  reality  modeling  can  pro¬ 
vide?  First  of  all,  a  virtual  reality  model  offers  a  risk-free  environment.  It  is  a  potential  hazard 
to  use  specimens  which  have  not  been  screened  for  Aids  and  Hepatitis.  Obviously,  the  appear¬ 
ance  and  odor  of  the  specimen  will  not  be  attractive  to  work  with  for  an  extended  period  of 
time.  In  addition,  it  will  deteriorate  and  become  spoiled,  thus  losing  its  anatomical  composi¬ 
tion  and  physiological  properties.  Even  using  the  best  and  most  sophisticated  mechanical  sim¬ 
ulators,  not  all  physiological  functions  can  be  reproduced  on  cadaver  specimens.  In  contrast, 
the  virtual  reality  biomechanical  models,  after  proper  validation,  can  provide  infinite  combi¬ 
nations  of  normal  and  pathological  conditions  to  study  their  effects  on  system  responses  and 
to  predict  therapeutic  outcomes.  Of  course  it  will  be  difficult  to  simulate  blood,  haptic  and 
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sensory  feedback.  These  are  not  mandatory  in  all  surgical  simulators  to  provide  practical  and 
essential  applications. 

The  flight  and  vehicular  simulators  are  the  best  engineering  examples  which  have  been  in 
extensive  use  during  the  last  two  decades.  They  serve  as  standard  test-beds  or  virtual  laborato¬ 
ries  for  personnel  training,  equipment  design  and  testing,  accident  simulation  and  prevention. 
The  same  simulators  could  be  created  to  study  human  musculoskeletal  system  function,  dis¬ 
ease  progression,  and  surgical  reconstruction  entirely  on  computer  models  —  a  basic  motiva¬ 
tion  to  develop  the  Virtual  Human.  Such  a  model,  however,  must  satisfy  certain  basic 
requirements  in  order  to  be  used  as  a  viable  surgical  simulator.  It  must  provide  anatomic  accu¬ 
racy  and  visual  fidelity.  In  certain  models,  organ  and  tissue  properties  must  be  provided.  For 
cardiovascular  and  pulmonary  function,  blood  flow  and  air  exchange  in  the  heart  and  lungs 
must  be  visualized.  In  the  musculoskeletal  system,  biomechanical  analysis  capability  should 
be  included  to  show  how  the  system  moves,  what  the  internal  forces  are,  and  how  they  inter¬ 
act.  Tactile  feedback  can  be  useful  through  instrument  design  and  virtual  environment  cre¬ 
ation. 

D.4  Development  of  the  Virtual  Human  Model 

Can  a  Virtual  Human  model  be  developed  to  study  the  normal  and  abnormal  functions  of  the 
musculoskeletal  joint  system?  The  answer  is  definitely  yes!  We  have  generated  a  computer 
graphic  model  of  the  human  musculoskeletal  system  that  can  perform  dynamic  sporting  activ¬ 
ities  such  as  baseball  pitching.  From  such  a  model,  one  can  study  the  joint  motion  and  forces 
involving  the  shoulder,  and  proceed  to  investigate  the  effect  of  fatigue  on  muscle  control  as  a 
potential  source  of  ligamentous  injury.  Many  other  applications  to  be  described  later  will 
clearly  demonstrate  the  unlimited  potential  of  such  technology. 

To  accomplish  the  required  simulation  goals,  a  three-dimensional  database  for  visualization  of 
human  anatomy  and  physiology  was  initiated  through  the  Advanced  Technology  Program 
funding  awarded  from  the  Commerce  Department  four  years  ago.  The  Virtual  Human  model¬ 
ing  work  and  its  medical  application  were  regarded  as  high-risk  technology  and  thus  rendered 
special  support.  Fresh  cadaver  bodies  were  obtained  for  MRI  and  CT  scans  at  refined  incre¬ 
ments.  These  bodies  were  later  frozen  and  embedded  in  foam  material  for  cryo-sectioning  at 
2-5  mm  intervals.  Based  on  the  digital  images,  bone  and  muscle  boundaries  were  obtained  to 
reconstruct  the  three-dimensional  models  using  patched  C-spline  functions.  When  the  bound¬ 
aries  of  the  vital  structures  and  tissues  such  as  nerves  and  vessels  were  not  clear  enough, 
stained  sections  were  used  to  distinguish  the  margins  with  the  aid  of  anatomy  text  books.  In 
many  aspects,  this  effort  was  similar  to  the  “visible  human”  program  sponsored  by  the 
National  Library  of  Medicine.  However,  the  major  difference  between  our  “Virtual  Human” 
and  the  “Visible  Human”  database  is  that  the  former  is  an  analytically  based  model  which  can 
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provide  biomechanical  analysis  with  results  animated  to  emulate  musculoskeletal  system 
function  with  remarkable  accuracy  and  flexibility. 

The  virtual  human  model,  or  the  Virtual  Biomechanical  Model,  has  many  unique  features  and 
capabilities.  This  model  is  visible,  anatomic,  and  interactive.  Muscle  orientation,  joint  geome¬ 
try  and  material  properties  can  be  changed  to  simulate  a  wide  variety  of  pathological  condi¬ 
tions.  Static  and  dynamic  analyses  can  then  be  performed  to  examine  their  effects  on  joint 
function.  Most  importantly,  joint  contact  pressure,  ligamentous  tension  and  the  stresses  in  the 
bone  can  all  be  determined  using  Finite  Element  Analysis  software  or  the  simplified  Rigid 
Body  Spring  Modeling  (Discrete  Element  Analysis)  technique.  Hence,  this  model  is  not  only 
anatomic  but  also  computational  (Figure  D-1).  With  kinematics  input,  measured  external  load¬ 
ing  and  estimated  inertial  properties  of  the  body  segments,  the  virtual  human  model  can  be 
made  to  perform  dynamic  activities  with  full  visualization  capability.  Its  interactive  nature 
provides  the  basic  requirement  for  a  surgical  simulator  to  fulfill  its  utility  in  medical  educa¬ 
tion,  research  and  patient  care. 


Figure  D-1:  The  flow-chart  diagram  for  the  Virtual  Musculoskeletal  model  for  biomechanical 
analysis. 
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D.5  Application  of  Virtual  Reality  Modeling 

Surgical  simulation  includes  three  fundamental  areas.  The  first  area  involves  a  surgery-based 
simulation  model  to  be  used  for  pre-operative  planning,  intra-operative  guidance  and  post¬ 
operative  outcome  assessment.  The  second  area  of  simulation  is  to  provide  an  analytical 
model  for  biomechanical  analyses  of  the  musculoskeletal  system  in  action,  a  virtual  dynamic 
testing  laboratory.  Finally,  a  surgical  simulator  using  both  graphical  and  physical  models  with 
haptic  and  sensory  feedback  can  provide  surgical  training  and  demonstration.  One  of  the 
examples  of  the  last  simulation  application  is  in  the  field  of  arthroscopic  surgery.  Using  simu¬ 
lated  fiber  optics  instruments,  the  surgeon  or  trainee  can  look  through  the  scope  to  locate  sur¬ 
gical  instruments  in  reference  to  vital  tissues  in  the  joint  space  and  perform  surgery  under 
direct-image  guidance. 

Another  example  is  a  three-dimensional  bony  structure  of  the  pelvis  providing  detailed  anat¬ 
omy  of  the  SI  joint,  pubic  symphysis,  the  hip  joints  and  all  the  essential  soft  tissues  defined  in 
geometric  and  physical  terms.  Free-body  analysis  in  order  to  produce  biomechanical  results 
based  on  applied  forces  and  measured  kinematic  input  data  can  be  carried  out.  With  this 
unique  capacity,  one  can  investigate  pelvic  instability,  hip  joint  force  transmission,  and  the 
effect  of  fracture  fixation  or  pelvic/acetabular  osteotomies  on  the  functional  responses  of  the 
hip  joint.  In  addition,  dynamic  analysis  of  the  hip  joint  during  gait  can  be  studied  under  simu¬ 
lation  of  abnormal  muscle  function.  This  model  can  be  used  to  estimate  the  joint  range  of 
motion  during  activities  of  daily  living  and  under  various  reconstructive  procedures,  including 
total  joint  replacement.  With  the  help  of  FE  analysis  and  the  Rigid  Body  Spring  Modeling 
technique,  three-dimensional  stresses  on  the  joint  surface  and  in  the  stem,  cement  and  bone 
cortex  can  all  be  calculated  as  a  function  of  loading  cycle  during  gait.  Such  a  model  also  offers 
an  objective  assessment  of  rehabilitation  programs  designed  for  the  related  joints  to  assure 
efficacy  and  safety. 

The  same  technology  presented  in  this  paper  can  be  applied  equally  well  in  non-surgical  disci¬ 
plines.  The  virtual  reality  modeling  technique  is  particularly  attractive  in  focalized  radiation 
for  the  control  of  cancer.  An  individual  patient's  lesion  location  can  be  scaled  on  the  existing 
graphic  model  to  plan  the  radiation  program.  Computer-aided  rehabilitation  (CAR)  and  the 
design  and  application  of  rehabilitation  robots  are  additional  examples.  Through  visual  feed¬ 
back,  exercise  can  be  modeled  and  analyzed  to  improve  rehabilitation  safety  and  effectiveness 
tailored  to  each  patient's  needs  and  limitation.  The  computer  model  of  the  exercised  limb  and 
joint  system  displayed  on  the  monitor  provides  quantitative  feedback  for  the  patient  to  pace 
their  effort  level  in  order  to  achieve  the  therapeutic  goal. 
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D.6  Discussion 

To  achieve  the  aforementioned  goals  for  surgical  simulation  using  virtual  reality  biomechani¬ 
cal  models,  certain  technical  obstacles  must  be  overcome.  The  virtual  reality  model  itself  has 
to  be  validated.  The  simulator's  reliability  and  consistency  must  be  established  and  we  must  be 
able  to  determine  the  ideal  type  of  simulator  for  specific  applications  in  surgical  training,  basic 
research  and  in  clinical  patient  care.  In  addition,  the  simulator  must  be  safe,  relevant,  practical 
and  most  importantly,  affordable.  For  preoperative  planning  purposes,  the  musculoskeletal 
model  must  be  compared  with  an  actual  anatomic  specimen  tested  on  mechanical  simulators 
under  similar  loading  conditions.  Model  validation  is  the  essential  prerequisite  for  such  tech¬ 
nology  to  gain  general  acceptance  in  the  medical  community.  Of  course,  there  is  no  substitute 
for  clinical  trials  in  order  to  establish  the  credibility  and  accountability  before  widespread 
clinical  application. 

With  the  enormous  technology  available  to  us  today,  computer-assisted  surgery,  tele-medi¬ 
cine,  and  tele-surgery  are  not  only  possible,  they  have  already  been  implemented  at  several 
institutions  for  research  and  demonstration  purposes.  Surgery  can  be  performed  at  a  remote 
site  under  tele-guidance  and  tele-intervention.  Practicing  tele-medicine  will  require  careful 
implementation  of  software  programs  incorporating  artificial  intelligence  to  facilitate  decision 
making  proficiency.  Additional  factors  involving  personnel  experience,  training  background, 
environmental  constraints,  the  extent  of  disease  or  traumatic  injury,  the  availability  of  facili¬ 
ties  and  instruments,  and  the  data/image  communication  network  efficiency  are  the  key  deter¬ 
minants  for  their  suecessful  and  practical  utilization.  Until  these  technical  barriers  are 
sufficiently  resolved,  much  of  the  advanced  medical  applications  discussed  here  will  remain 
academic. 

Robotics  and  computer-assisted  medical  intervention  require  certain  enabling  technologies. 
Much  of  these  enabling  technologies  are  readily  available,  but  not  appreciated  by  the  medical 
professionals  and  the  policy  making  administrators.  It  is  up  to  us  to  transfer  these  available 
technologies  from  engineering  and  physical  science  domains  into  medicine  and  surgery  while 
attempting  to  convince  the  funding  agencies  and  foundations  of  the  enormous  potential  of 
these  new  developments.  A  combination  of  a  virtual  reality  computer  model  and  a  physical 
model,  a  form  of  Hybrid  Reality,  can  be  utilized  to  overcome  the  inherent  skepticism  among 
physicians  and  basic  scientists.  We  have  to  take  advantage  of  the  innovative  imaging  technol¬ 
ogy  developed  for  the  mass  media  and  the  entertainment  industry  to  provide  a  more  vivid 
graphic  model  and  the  means  to  manipulate  and  scale  the  model  in  order  to  adopt  a  patient's 
specific  conditions  to  the  existing  virtual  model  for  the  required  analyses  and  visualization. 
Although  these  are  formidable  tasks,  these  challenges  can  be  met  by  well-trained  engineers 
with  dedicated  effort  towards  medicine  and  surgery,  as  they  have  done  so  for  many  medical 
and  non-medical  fields  in  the  past.  With  appropriate  funding,  it  will  only  be  a  matter  of  time 
before  such  technology  seeks  its  proper  place  in  the  vast  medical  world. 


66 


Section  D  -  Surgical  Simulator  and  Virtual  Reality  Modeling  in  Orthopaedics 


Engineering  technology  has  not  been  blamed  for  today’s  escalating  medical  costs.  However, 
unwise  or  careless  application  of  the  technology  discussed  herewith  may  affect  the  remarkable 
reputation  established  by  the  bioengineers  through  their  enormous  contributions  to  the  medi¬ 
cal  field  during  the  long  and  productive  era  covering  the  last  three  decades.  Our  optimal  goal 
is  to  ensure  that  modem  technology  and  medical  research  accomplishments  can  be  cost  effec¬ 
tively  applied  in  today’s  managed  care  environment  through  engineering. 

Finally,  when  discussing  the  practical  nature  of  RCAMI,  the  non-technical  issues  involved 
must  also  be  carefully  elucidated.  If  such  technology  is  contemplated  for  clinical  application, 
it  must  be  assessed  and  approved  by  the  consumers,  i.e.,  the  patients.  Its  validity  should  not  be 
evaluated  by  anyone  directly  involved  in  the  development  or  transfer  of  such  technology.  The 
balance  between  cost  and  benefit  must  be  carefully  weighed.  Acceptance,  not  only  by  the 
medical  professionals  and  the  health  care  management  but  also  by  the  lay  people,  will  provide 
the  required  seal  of  approval.  The  FDA  accreditation  and  the  annoying  liability  issue  are  the 
additional  concerns.  Until  then,  the  entire  field  related  to  RCAMI  will  remain  in  the  academic 
domain.  Even  so,  such  development  does  represent  a  landmark  advancement  in  fostering  engi¬ 
neering  technology  in  medical  research  and  education.  This  will  be  the  most  powerful  tool  for 
all  human  health  related  endeavors  for  the  next  century. 
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Appendix  F  Safety  Issues  Presentation 

Presentation  given  by:  Brian  Davies 

There  is  a  need  to  move  towards  an  international  consensus  on  how  safe  systems  must  be: 

•  To  protect  patients; 

•  To  protect  medical  personnel  (from  litigation  as  well  as  physically); 

•  To  re-assure  system  manufacturers/suppliers 

Many  of  the  points  can  only  be  completely  resolved  in  detailed  discussions  of  the  specific  pro¬ 
cedure.  However,  hopefully  it  will  be  possible  to  get  enough  information  to  set  out  some  gen¬ 
eral  guidelines. 

The  area  seems  to  break  down  into  the  following  aspects: 

•  Traditional  safety-critical  systems  issues  (but  considered  from  the  special  standpoint  of 
RCAMI). 

-  Should  software  be  developed  (as  was  suggested  by  UK  Health  and  Safety  Executive 
at  one  stage)  by  three  different  software  groups  in  three  different  languages  and  run 
on  different  hardware  so  that  motions  are  made  only  when  all  three  agree? 

-  Should  systems  be  programmed  in  Ada,  C-i-i-,  any  object  oriented  code? 

-  Can  a  standard  operating  system  be  used  (e.g.,  DOS)  or  must  it  be  specially  config¬ 
ured? 

-  Is  it  OKto  use  standard  computer  hardware  (e.g.,  a  PC)  or  must  it  be  ruggedized  / 
modified? 

-  Should  formal  methods  always  be  used? 

-  Should  Failure-Mode-Effect  Analysis  always  be  used? 

-  Should  Fault-Tree  Analysis  always  be  used? 

-  Should  a  separate  emergency  power  supply  always  be  supplied? 

-  Should  a  “positive  polling”  electronic  communication  Bus  be  used  to  ensure  integ¬ 
rity  of  sub  systems  and  their  communication?  (e.g.,  of  the  type  suggested  by  Impe¬ 
rial  College  in  the  European  TIDE  Project  “M3S-Multiple  Master  Multiple  Slave”  in 
which  a  CAN  Bus  was  supplemented  by  hardwired  Key  and  “Dead-Man”  Switch 
lines). 

•  pre-operative  imaging,  modeling  and  planning  (applies  to  all  systems;  CAS  or  robotic) 

-  What  measures  are  provided  to  ensure  correctness  and  accuracy  throughout  (e.g., 
that  the  imaging  and  3D  modeling  are  optimal  for  the  task)? 
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-  Are  fiducial/anatomical  markers  identified  throughout  and  available  for  subsequent 
registration? 

-  What  pre-operative  planning  aspects  are  necessary  to  verify  that  the  procedure  is 
correct  (e.g.,  that  the  displayed  plan  and  the  intended  plan  are  the  same,  or  that  the 
simulation  correlates  with  reality)? 

-  Are  all  assumptions  built  into  the  imaging/modeling  clearly  identified  for  the  sur¬ 
geon  (e.g.,  regions  where  accuracy  is  diminished  or  tool  motion  is  limited)? 

•  Issues  associated  with  the  use  of  a  live  (active)  robot  next  to  people,  (Includes  Tele-opera- 
tors) 

-  Lack  of  relevance  of  guidelines  from  industrial  robotics  (which  discourage  use  next 
to  people) 

-  Should  special  purpose  systems  for  the  task  generally  be  encouraged  (e.g.,  of  force 
levels,  reach,  and  motions  just  adequate  for  the  task),  rather  than  slight  modifications 
to  industrial  robots? 

-  Should  force  sensors  be  added  for  each  type  of  joint  to  trip  if  force  exceeded  (and/or 
could  they  just  be  motor  current  sensors)? 

-  Should  a  mechanical  constraining  system  be  added  to  limit  motions  to  a  safe  region 
if  all  software  safety  fails? 

-  What  safety  software  and  sensor  systems  are  desirable  (e.g.,  should  two  position  sen¬ 
sors  be  provided  per  joint,  one  at  the  motor  and  one  at  the  joint  output)? 

-  For  emergencies,  should  a  “Dead-Man”  switch  approach  be  provided  or  should  an 
emergency  “ofF’  switch  be  held  by  the  surgeon?  Alternatively,  is  it  adequate  to  sim¬ 
ply  have  an  “off’  switch  in  the  vicinity? 

-  Should  the  system  always  have  a  high  reliability  or  is  it  enough  that  the  system 
always  fails  to  a  safe  condition? 

•  Issues  of  powered  robots  used  passively  (e.g.,  to  carry  a  fixture  or  jig  which,  in  turn,  is 
used  by  the  surgeon). 

-  Should  power  be  removed,  and  brakes  be  engaged,  before  surgeon  is  in  vicinity? 

-  Is  it  allowable  for  the  surgeon  to  hold,  say,  a  force  sensor  on  the  wrist  and  lead  the 
robot  to  the  final  position? 

-  Is  it  allowable  for  a  large  reduction  gearbox  to  be  added  to  the  drives  of  the  robot,  to 
slow  down  the  motion  (recognizing  that  this  increases  the  force  levels  proportion¬ 
ately)  and  if  so,  how  close  to  the  patient  can  it  be  positioned? 

•  Computer  Assisted  surgery 

-  Should  undesired  motions  of  the  surgeon  (e.g.,  tremor,  or  motion  into  prohibited 
regions)  be  physically  restricted? 

-  Should  only  a  warning  be  given? 
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-  Should  localizers  always  give  warnings  when  moving  into  areas  of  reduced  accu¬ 
racy,  or  is  it  adequate  to  stop  the  procedure  (e.g.,  when  a  camera-based  sensor  is  out 
of  range  or  a  manipulator  arm  is  at  its  limit  of  stroke)? 

•  Tool  registration  at  the  start,  and  during,  the  procedure  (i.e.,  of  tool  tip  in  camera  based 
tracking,  or  of  manipulator  tip  in  arm  based  tracking,  or  of  robot  end  point  for  robotic 
tracking). 

-  What  methods  should  be  used  to  ensure  optimal  accuracy? 

-  What  warnings  should  be  given  to  surgeons  about  conditions  which  lead  to  poor 
accuracy  or  reliability? 

-  Is  force  control  necessary  for  accurate  registration? 

-  Is  it  preferable  to  use  fiducials  screws,  anatomical  markers,  or  plaster  casts  contain¬ 
ing  fiducials? 

-  How  best  to  register  points  remote  from  the  area  of  interest? 

-  How  best  to  register  soft  tissue? 

•  Intra-operative  tracking  of  target. 

-  In  orthopaedics,  should  patient  be  rigidly  clamped  with  a  warning  only  if  there  is 
inadvertent  motion,  or  should  patient  motion  be  allowed  with  dynamic  referencing 
(i.e.,  the  target  location  updated  on-line)? 

-  What  features  are  required  to  verify  that  all  is  ok  and  incorrect  advice  is  not  being 
given?  (unlike  pre-operative  planning,  it  is  usually  not  possible  to  intra-operatively 
conduct  tests  of  correctness) 

-  In  soft  tissue,  how  should  motion  be  tracked  or  compensated  for? 

-  Can/should  change  of  soft  tissue  shape,  when  cut  or  pressed,  be  taken  into  account? 

-  Should  a  dynamic  referencing  system  have  two  separate  means  of  measurement? 

•  HCI 

-  How  much  information  should  be  given  to  surgeons  (pre  and  intra-operatively)? 

-  In  what  form  is  it  best  displayed? 

-  Should  step-by-step  instructions  always  be  given,  (e.g.,  sterilization  sequence)? 

-  Should  surgeons  have  full  awareness  of  limitations  (e.g.,  of  range  of  motions  and 
forces)? 

-  Should  surgeons  have  full  awareness  of  best  practice  in  using  systems  (e.g.,  of 
regions  of  most/least  accuracy)? 

-  Should  surgeons  be  aware  of  current  status  of  all  targets  and  tools  and  of  percentage 
of  plan  completed,  etc.? 

-  Should  all  motions  and  instructions  automatically  be  recorded? 

-  In  an  emergency,  what  information  is  vital  to  display  and  what  is  confusing? 
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•  Interruptions. 

-  If  procedure  is  interrupted,  what  verification  is  required  to  ensure  all  is  still  correct? 

-  Should  system  be  repositioned  at  start  to  re-register  if  interrupted? 

-  Should  system  simply  pause  before  continuing? 

-  Should  robots  always  re-play  last  motion  sequence? 
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Computer-Aided  Surgery  -  Presentation 

Presentation  given  by:  Kirby  G.  Vosburgh 

New  Technologies  follow  three  steps  in  their  development: 

•  Creation 

•  Validation 

•  Utilization 

Expanding  this  description  for  the  technologies  comprising  high  technology  surgery: 

G.l  Creation 

The  concept  of  aiding  and  enhancing  the  performance  of  physicians  in  interventional  proce¬ 
dures  has  been  created  over  the  past  decade.  Diverse  streams  of  research  have  been  blended  to 
create  this  new  field  including: 

-  New  surgical  approaches,  such  as  endoscopy  and  semi-automated  microscopy 

-  Advances  in  interventional  radiology 

-  Advances  in  imaging 

-  Advances  in  computational  support 

-  Applications  of  robotics  technology  to  surgery 

These  elements  impact  all  stages  of  treatment;  in  planning  guidance,  monitoring,  and  control. 

G.2  Validation 

This  is  the  current  stage  of  many  of  the  concepts  in  the  field.  As  will  all  technologies  applied 
to  medical  care,  the  validation  process  has  two  principal  elements: 

-  Demonstration  of  Technical  Performance  -  This  includes  the  effort  to  “show  that  it 
works”  and  demonstrating  first  order  safety  and  creating  a  clinical  test  system. 

-  Clinical  Viability  -  Here,  the  new  technique  is  demonstrated  to  be  clinically  effica¬ 
cious,  safe,  reasonable  in  cost,  and  generally  superior  to  current  practice.  Many  engi¬ 
neers  underestimate  the  extent  and  difficulty  of  this  stage,  and  fail  to  appreciate  the 
care  required  to  structure  a  sound,  convincing  clinical  trial. 


79 


Appendix  G  -  Technology  Transfer  for  Computer-Aided  Surgery  -  Presentation 


From  an  engineering  perspective,  the  systems  used  in  clinical  validation  will  be 
more  sophisticated  than  “production  prototypes”.  They  are  designed  to  perform 
focused  experiments  to  answer  key  questions  pacing  the  structure  of  the  clinical  tri¬ 
als.  Then,  in  the  later  stages,  to  be  sufficiently  robust  to  support  tests  with  human 
subjects,  while  also  being  sufficiently  flexible  to  permit  optimization  of  the  tech¬ 
nique. 


G.3  Utilization 

When  the  technique  is  validated,  commercial-quality  systems  may  be  designed.  As  computer- 
based  systems  and  closed-loop  automation  become  more  widely  used  in  medical  care, 
(expanding  from  their  current  base  in  diagnosis  and  planning),  a  high  standard  or  reliability 
must  be  maintained. 

Thus,  to  the  maximum  extent  possible,  the  new  technique  should  be  simple  and  robust  —  pre¬ 
optimized  for  its  task  —  so  that  the  physician  cam  concentrate  on  the  case  of  the  patient,  not  on 
the  operation  of  the  machine. 

G.4  Some  Positive  Steps 

As  shown  at  this  workshop,  several  factors  are  manifest  to  support  early  success: 

-  Consistent  Long  Term  Goals  -  This  work  is  characterized  by  the  commitment  and 
staying  power  of  the  medical  and  engineering  leadership.  They  key  issues  are  being 
understood  in  common,  and  being  addressed  with  rigor. 

-  Standardization  -  There  is  progress  on  standardizing  terminology,  so  that  commu¬ 
nication  among  workers  (who  represent  different  training,  applied  to  a  variety  of 
professional  disciplines,  and  working  in  a  range  of  professional  environments)  may 
join  forces  effectively. 

-  Technology  Differentiation  -  Some  enabling  technologies,  such  as  thermal  surgery 
devices  or  image  segmentation  algorithms,  are  unique  to  the  field.  Others,  such  as 
computers,  are  commercial  units.  Construction  of  a  system  then  involves  the  integra¬ 
tion  of  unique  and  commodity  parts.  Progress  will  accelerate  to  the  extent  that  the 
commercial  parts  assume  as  much  function  as  possible,  with  the  simplest  practical 
standardized  interfaces.  Benefits  of  increasing  the  commercial  content  of  the  system 
will  include  lower  cost,  simpler  upgrading  and  improved  safety,  reliability,  maintain¬ 
ability,  etc. 

-  End-User  Pull  -  The  end-user  of  a  technology  (in  this  case,  surgeons  and  other  med¬ 
ical  interventionists)  are  in  the  best  position  to  set  goals,  and  they  are  also  in  the  best 
position  to  spot  small  “bumps  in  the  road”  that  slow  progress.  The  community  of 
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leaders  of  the  technology,  as  represented  by  the  participants  in  this  Workshop,  is  then 
able  to  smooth  the  bumps  efficiently.  In  particular,  clinical  leaders  will  “pull”  appli¬ 
cations  more  smoothly  than  technologists  will  “push”  them. 

-  Bridging  the  “Valley  of  Death”  -  In  the  development  of  many  technologies,  there  is 
a  gap  in  funding  and  institutional  support  as  the  research  phases  down,  but  before  the 
rise  of  commercial  development.  As  noted  by  Col.  Satava,  many  good  ideas  die  in 
this  phase,  where  the  practical  challenges  appear  to  be  neither  interesting  to  the 
research  funding  community  nor  profitable  to  the  businessman.  We  can  bridge  this 
gap  by  maintaining  interest,  focusing  our  efforts  on  attractive  demonstration 
projects,  and  stimulating  early  clinical  participation. 
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Appendix  H  Image-Guided  Tlimor 
Diagnosis  and  Treatment 

Presentation  given  by:  Faina  Shtem 

Over  the  last  few  years,  image-guided  computerized  interventions,  such  as  laparoscopic, 
endoscopic,  endovascular  and  radiologic,  emerged  as  a  minimally  invasive  alternative  to 
major  surgical  procedures.  The  interest  at  the  National  Cancer  Institute  stems  from  the  poten¬ 
tial  impact  of  computerized  interventions,  such  as  tumor  biopsy  and  treatment,  on  the  practice 
of  oncology,  radiology  and  surgery,  when  major  surgical  procedures  may  be  replaced  by  min¬ 
imally  invasive,  cost-effective  ambulatory  interventions. 

In  contrast  to  conventional  surgery,  imaging-guided  interventions,  which  minimize  tissue 
damage  associated  with  surgical  visualization  and  access,  have  the  potential  to  reduce  patient 
suffering,  morbidity  and  mortality,  recovery  period,  and  hospitalization  costs. 

The  clinical  impact  of  computer-assisted  procedures  was  shown  in  neurosurgery,  where  addi¬ 
tion  of  CT  imaging  reduced  morbidity  of  stereotactic  brain  biopsy  from  30%  to  less  that  1%. 
More  recently,  image-guided  percutaneous  stereotactic  breast  needle  biopsy  has  been  demon¬ 
strated  as  a  minimally  invasive,  cost  effective  alternative  to  diagnostic  open  surgical  excision 
for  patients  with  non-palpable  lesions.  In  contrast  to  open  surgical  biopsy,  stereotactic  breast 
biopsy  is  a  simple,  automated  procedure,  which  causes  no  pain  or  disfigurement.  The  cost  of 
stereotactic  procedures  is  about  28%  of  that  for  surgical  biopsy.  Even  conservatively  esti¬ 
mated,  presuming  that  only  35  to  50%  of  surgical  biopsies  will  be  replaced  by  stereotactic 
procedures,  the  annual  national  cost  savings  are  predicted  at  about  0.4  to  0.8  billion  dollars. 

Minimally-invasive  procedures,  however  diverse,  have  two  common  features:  1)  digital 
images;  and  2)  remote  manipulations.  In  a  subjective,  arbitrary  manner,  digital  images  may  be 
divided  in  two  major  groups.  Video  camera-based  images  may  be  generated  during  endo¬ 
scopic,  laparoscopic  and  some  endovascular  interventions,  when  field  of  view  is  limited  in 
size  and  visualization  is  limited  to  tissue  surfaces.  Radiologic  modalities,  on  the  other  hand, 
are  unique  in  their  ability  to  provide  subsurface  three-dimensional  visualization,  with  large 
field  of  view. 

As  opposed  to  digital  image-based  synthetic  reality,  virtual  reality  is  a  computer  simulation  of 
surgical  experience.  Virtual  reality  may  take  image-based  synthetic  reality  one  step  further, 
when  visualization  may  be  combined  with  simulation  of  other  surgical  experiences,  including 
manual,  vestibular,  kinesthetic  [and  even  olfactory]  influences. 

Digital  image  display,  in  turn,  creates  a  possibility  of  a  remote  manipulation,  which  is  essen¬ 
tially  an  interaction  of  a  surgeon  with  internal  tissues  via  computerized  reality,  rather  than 
direct  surgical  access  and  visualization.  In  essence,  the  computer  workstation  becomes  a 
patient  substitute. 
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One  can  imagine  that  interactive  surgical  computer  workstations,  including  virtual  reality  sys¬ 
tems,  may  be  interfaced  with  telerobotics,  such  as  for  instance  robotic  arm,  which  may  follow 
human  movement  during  intervention  with  great  precision.  Such  an  interface  would  then 
allow  for  surgery  either  a  few  feet  away,  or  a  few  hundred  miles  away.  Of  course,  real  time 
telesurgery  raises  a  number  of  important  questions  of  high  performance,  low  cost  telecommu¬ 
nications  of  the  future. 

While  we  are  on  the  subject  of  the  “futuristic”  possibilities,  image-guided  robotics-driven  sur¬ 
gery  comes  to  mind.  The  potential  role  of  robotics  in  image-guided  interventions  is  quite 
intriguing  for  several  reasons,  including  their  potential  to  improve  human  dexterity.  Recently, 
an  image-guided  robot  has  been  used  for  retinal  surgery  and  orthopaedic  procedures,  and 
other  robotic  applications  are  under  development. 

Future  trends  in  image-guided  computerized  interventions  may  include:  1)  Real  time  3D  visu¬ 
alization;  2)  Virtual  Reality;  3)  Telesurgery;  4)  Robotics;  and  5)  Telecommunications.  Based 
on  the  future  promise  and  exciting  vision  for  this  field,  NCI  is  currently  considering  a  joint 
program  with  NASA,  Ballistic  Missiles  Defense  Organization  and  the  Society  of  Cardiovas¬ 
cular  and  Interventional  Radiology  entitled  “Technology  Transfer  in  Image-Guided  Tumor 
Diagnosis  and  Treatment”. 

The  goal  of  this  effort  is  to  advance  the  current  state-of-the-art  in  image-guided  tumor  diagno¬ 
sis  and  treatment  through  transfer  of  defense,  intelligence  and  space  technologies.  In  order  to 
achieve  this  goal,  we  intend  1)  to  review  current  fundamental  technologic  roadblocks;  2)  to 
develop  a  technologic  problem  statement;  3)  to  conduct  a  national  search;  and  4)  to  evaluate 
the  applicability  and  clinical  relevance  of  the  identified  defense,  intelligence  and  space  tech¬ 
nologies.  We  created  eight  technologic  teams,  covering  such  areas  as  3-D  Imaging,  intra-oper¬ 
ative  guidance,  real  time  imaging,  surgical  workstation  design,  telesurgery,  virtual  reality, 
telerobotics,  and  telecommunications.  The  next  step  will  be  to  complete  a  technologic  prob¬ 
lem  statement  for  four  specific  oncology  areas:  Head  &  Neck  Tumors;  Breast  Cancer;  Prostate 
Cancer;  and  Liver  Metastases.  This  problem  statement  will  be  circulated  to  over  250  defense, 
intelligence  and  space  laboratories. 

In  summary,  the  art  of  radiologic  imaging,  when  combined  with  computer  science,  robotics 
and  advanced  information  technologies  may  redefine  the  practice  of  medicine,  oncology  and 
surgery.  Surgical  interventions  may  become  more  safe  and  effective  in  order  to  reduce  pain 
and  suffering,  to  improve  quality  of  life.  We  hope  that  the  national  program  currently  under 
discussion  between  NCI,  other  government  agencies  and  the  academic  community  will  make 
a  modest  contribution  to  the  achievement  of  this  future  vision. 
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The  Robotics  and  Computer  Assisted  Medical  Interventions  (RCAMI)  workshop  will  be 
divided  into  four  working  groups,  each  with  its  own  area  of  focus.  The  goal  of  each  working 
group  will  be  to  examine  issues,  needs  and  requirements  related  to  a  particular  RCAMI  appli¬ 
cation  area,  and  to  identify  areas,  tasks,  and  problems  for  future  research  and  development. 
This  pre-workshop  report  offers  an  initial  framework  for  discussion  by  defining  the  four  appli¬ 
cation  areas,  and  a  variety  of  technical  and  non-technical  issues  which  span  each  of  the  appli¬ 
cation  areas.  In  addition,  within  each  application  area  a  number  of  example  systems  and  issues 
are  presented  in  an  attempt  to  differentiate  the  four  application  areas. 

The  four  RCAMI  working  groups  will  be  charged  with  investigating  issues  related  to  the  fol¬ 
lowing  application  areas: 

1 .  Surgical  Simulation  -  including  physical-based  modeling,  biomechanical  simulation, 
visual  and  haptic  interaction;  for  use  in  applications  such  as  teaching,  training,  pre-treat¬ 
ment  planning  and  rehearsal. 

2.  Image  Guided  Procedures  -  the  intra-operative  use  of  computers,  sensors,  graphics,  etc. 
to  assist  or  guide  a  surgical  procedure,  usually  based  upon  pre-operative  medical  images. 
Does  not  include  active  or  semi-active  robotic  systems. 

3.  Robotics/Manipulators  -  the  intra-operative  use  of  active  or  semi-active  robotic/manipu¬ 
lation  systems  to  assist  in  image-guided  surgical  procedures. 

4.  Teleinterventions  -  the  use  of  remote  information  and/or  guidance  in  medical  interven¬ 
tions  involving  the  transduction  and  transmission  of  perceptions  and/or  actions  at  a  dis¬ 
tance. 

In  addition  to  the  four  application  areas,  five  enabling  technologies  have  been  identified  which 
span  the  four  application  areas.  The  enabling  technologies  provide  the  building  blocks  which 
are  necessary  for  development  within  the  application  areas.  Each  working  group  should  exam¬ 
ine  issues  related  to  the  intersection  of  the  enabling  technologies  with  the  group’s  application 
area.  Enabling  technologies  have  been  classified  into  the  following  categories: 

1 .  Augmented  or  hybrid  reality  displays  -  devices  which  permit  the  combination  of  real 
and  virtual  worlds  to  produce  an  enhanced  sense  of  realty. 

2.  Medical  imaging  technologies  (e.g.,  X-Ray,  CT,  MRi,  PET,  Ultrasound,  etc.)  -  anatomi¬ 
cal  and  functional  imaging  of  the  human  body  for  use  both  pre-  and  intra-operatively. 
Includes  novel  interventional  imaging  technologies. 

3.  Non-medical  sensing  and  imaging  technologies  -  (e.g,  position  estimation  sensors,  range 
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imagers,  video  cameras,  etc.)  -  for  accurate  and  reliable  measurement  of  patient  anatomy, 
surgical  tools,  surgeon’s  head  position,  etc. 

4.  Registration  methods  -  for  identifying  the  spatial  relationship  between  two  or  more  rep¬ 
resentations  of  the  same  structure  (e.g.,  a  pre-operative  CT  image  of  a  skull  to  multiple 
intra-operative  X-Rays  of  the  same  skull).  The  resulting  spatial  relationship  may  be  rigid 
or  deformable. 

5.  Medical  image  processing  and  understanding  -  the  low  level  manipulation  and/or  high 
level  interpretation  of  medical  images  to  extract  relevant  structures  and  information. 

There  are  three  technical  issues  which  can  not  be  classified  as  enabling  technologies,  and  yet 
have  implications  to  each  of  the  four  application  areas.  The  intersection  of  these  common 
technical  issues  with  each  application  area  should  also  be  explored  by  the  working  groups. 

1 .  System  validation  and  requirement  specification  -  the  validation  of  RCAMI  systems  in 
terms  of  accuracy,  usability,  robustness,  etc.  The  task-specific  determination  of  system 
requirements  in  terms  of  accuracy,  reliability  etc.  This  is  an  area  which  has  not  received 
sufficient  consideration  in  the  past,  and  will  require  increasing  attention  as  more  RCAMI 
systems  become  used  in  the  clinic. 

2.  Minimally  invasive  procedure  design  -  the  trend  in  surgery  from  procedures  with  large 
explorations  and  exposures  to  procedures  with  limited  access  and  restricted  direct  visibil- 
ity.  ^ 

3.  Design  for  safety  -  design  of  RCAMI  systems  to  ensure  safe  and  reliable  operation. 

Finally,  several  non-technical  issues  which  are  common  to  the  RCAMI  application  areas 
should  be  examined.  These  issues  include: 

1 .  Clinical  evaluation  and  cost  benefit  analysis  -  How  to  determine  if  new  technology 
improves  patient  outcomes?  What  is  the  meaning  of  improved  outcome?  Lower  costs? 
Reduced  complications?  Is  the  technology  economically  feasible  for  clinical  use? 

2.  Acceptance  -  Will  the  technology  be  readily  accepted  by  physicians,  hospital  boards, 
patients,  insurance  companies,  regulatory  bodies? 

3.  Regulatory  issues  -  Should  this  technology  require  sanctioning  from  government  bodies 
in  terms  of  licensing  and  training?  To  what  extent? 

4.  Liability  -  Who  is  liable  if  the  technology  fails? 

A  summary  of  the  application  areas  and  the  common  issues  /  technologies  is  presented  in 
Table  I- 1.  The  directive  of  each  working  group  is  to  explore  the  intersection  between  the 
applications  areas  and  the  issues  /  technologies  as  represented  by  the  shaded  region  of  the 
table.  Several  comments  regarding  the  framework  in  this  table  are  warranted.  First,  the  termi¬ 
nology,  definitions,  and  issues  implicit  in  the  framework  are  meant  as  a  starting  point  for  dis¬ 
cussion  by  the  working  groups.  Groups  should  feel  free  to  expand  upon  or  modify  this 
structure  as  they  choose.  Second,  there  is  inevitable  overlap  among  the  various  application 
areas.  Each  group  should  be  aware  of  the  domains  of  the  other  groups  in  order  to  avoid  dupli¬ 
cation  of  effort.  For  example,  while  pre-operative  planning  is  an  essential  component  of 
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Image-Guided  Procedures,  the  Surgical  Simulators  group  will  have  primary  responsibility  for 
issues  related  to  pre-operative  planning. 

An  additional  goal  of  the  workshop  will  be  to  generate  definitions  of  commonly  used  termi¬ 
nology  within  the  RCAMI  field.  An  initial  list  of  terms  requiring  formal  definition  is  pre¬ 
sented  below;  however,  working  groups  should  feel  free  to  suggest  additional  terms.  Each  of 
the  working  groups  has  been  assigned  a  set  of  terminology  to  define  as  specified  in  Table  2. 
The  terms  under  the  heading  “Common  Issues”  should  be  defined  by  each  of  the  working 
groups. 

What  follows  is  a  more  detailed  description  of  each  of  the  four  application  areas.  Key  technol¬ 
ogies,  example  systems,  and  basic  issues  are  presented  in  an  attempt  to  better  define  the  appli¬ 
cation  areas. 

I.l  Surgical  Simulators 

In  the  context  of  the  RCAMI  workshop,  the  application  area’s  of  surgical  simulators  include: 
physical-based  modeling,  biomechanical  simulation,  visual  and  haptic  interaction;  for  use  in 
applications  such  as  teaching,  training,  pre-treatment  planning  and  rehearsal. 

Some  of  the  key  tools  in  this  area  include  mathematical  modeling  techniques  such  as  finite 
element  analysis,  kinematic  analysis,  deformable  modeling  methods,  and  surgical  procedure 
optimization.  Other  relevant  tools  include  visualization  techniques,  computer  graphics,  haptic 
modeling  and  transduction,  and  anatomical  model  creation. 

Surgical  simulators  can  be  used  in  a  number  of  different  applications  including  [3]: 

1 .  Medical  education  -  for  instructing  students  in  areas  such  as  anatomy,  physiology,  and  bio¬ 
mechanics  (e.g.,  interactive  multimedia  presentations  of  anatomical  atlases). 

2.  Training  and  accreditation  -  developing  and  evaluating  skills  required  by  clinicians  for 
performing  particular  tasks  (e.g.,  laparoscopic  tools  usage,  spinal  tap  insertion).  These 
technologies  also  provide  a  mechanism  for  quantitatively  evaluating  surgical  skills  based 
upon  a  common  evaluation  metric. 

3.  Surgical  planning  -  for  pre-operative  decision  making  regarding  how  a  given  procedure 
will  be  performed.  May  incorporate  tools  for  predicting  and  optimizing  surgical  outcomes. 
Also  provides  the  ability  to  rehearse  a  procedure  and  refine  techniques. 

Physical-based  medical  simulators  have  been  compared  by  some  to  flight  simulators  which 
are  commonly  used  for  safely  training  pilots  [20].  Simulators  provide  a  risk-free  environment 
for  exposing  the  trainee  to  a  wide  variety  of  scenarios.  These  scenarios  may  include  infre¬ 
quently  occurring  complications  which  a  typical  student  might  never  otherwise  experience 
during  training.  Both  medical  and  flight  simulators  have  very  high  realism  requirements.  The 
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Application  Areas  /  Working  Groups 

Teleintervention 

Robotics  / 
Manipulators 

Image-Guided 

Procedures 

Surgical 

Simulators 

Augmented  /  Hybrid  Displays 

Medical  Imaging  Technologies 

Sensing  /  Imaging  Technologies 

Registration  Methods 

Medical  Image  Processing  and  Understanding 

System  Validation  and  Requirement  Specification 

Minimally  Invasive  Procedure  Design 

Safety 

Clinical  Evaluation  &  Cost  Benefit  Analysis 

Acceptance 

Regulatory  Issues 

Liability 

Other? 
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Surgical  Simulators 

•  Virtual  reality 

•  Surgical  simulator 

•  Physical  modeling 

•  Pre-treatment  planning 

Image-Guided  Procedures 

•  Computer-assisted  surgery 

•  Computer-controlled  surgery 

•  Image-guided  surgery 

•  Robot-assisted  surgery 

Robotics  /  Manipulators 

•  Medical  robot 

•  Powered  and  unpowered  robotic  arm 

•  Passive,  semi-active  and  active  robotic  arm 

•  Passive  and/or  Active  compliance 

•  Interventional  robotic  arm 

Teleintervention 


•  Telepresence 

•  Telemedicine 

•  Telesurgery 

•  Teleintervention 

•  Telemanipulator 

Common  Issues 


•  Registration 

•  Calibration 

•  Augmented  reality 

•  Hybrid  reality 

•  Minimally  invasive  procedure 


Table  2:  Terminology  to  define  within  application  areas 
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training  environment  must  be  sufficiently  similar  to  the  actual  environment  in  order  for  the 
simulator  to  be  beneficial. 

One  colleague  has  suggested  that  medical  simulators  should  possess  the  following  criteria  in 
order  to  be  convincing  [21]: 

1.  Fidelity  (high  resolution  graphics). 

2.  Organ  properties  (deformation  from  morphing  or  kinematics  of  joints). 

3.  Organ  reaction  (such  as  bleeding  from  artery  or  bile  from  the  gall  bladder). 

4.  Interactivity  (between  objects  such  as  surgical  instruments  and  organs). 

5.  Sensory  feedback  (tactile  and  force  feedback). 

General  issues  related  to  the  area  of  medical  simulators  include:  the  need  for  accurate  anatom¬ 
ical  modeling;  the  need  to  incorporate  patient  specific  information;  and  the  requirement  that 
simulations  must  eventually  be  validated  against  reality  to  ensure  effectiveness. 

Some  examples  of  surgical  simulation  systems  which  have  been  submitted  in  the  pre-work¬ 
shop  papers  include:  Georgia  Institute  of  Technology’s  eye  surgery  training  simulator  [22]; 
Delp’s  biomechanical  muscle  simulator  [7];  Delp  and  Rosen’s  Military  Medical  Trainer  [6]; 
and  Kaneko’s  ear  reconstruction  system  [16]. 

1.2  Image  Guided  Procedures 

In  the  context  of  the  RCAMI  workshop.  Image  Guided  Procedures  include  the  intra-operative 
use  of  computers,  sensors,  graphics,  etc.  to  assist  or  guide  a  surgical  procedure,  usually  based 
upon  pre-operative  medical  images.  Active  or  semi-active  robotic  systems  are  explicitly  omit¬ 
ted  from  this  application  area  as  they  are  addressed  in  the  Robotics/Manipulators  area. 

Image-guided  systems  typically  use  either  pre-  or  intra-operatively  acquired  medical  images 
for  assisting  the  surgeon  in  a  guidance  or  navigational  task.  During  surgery,  the  position  of 
surgical  tools  and  patient  anatomy  may  be  tracked  so  that  correspondence  can  be  established 
between  the  imagery  and  the  current  surgical  state.  By  relating  this  state  information  to  a  pre¬ 
operative  plan,  feedback  can  be  derived  and  presented  to  help  guide  the  surgeon  towards 
achieving  a  particular  goal. 

In  a  recent  paper  describing  an  image-guided  stereotactic  biopsy  system,  several  desirable 
properties  for  image  guided  systems  were  presented  [4]: 

1.  The  need  for  accuracy  and  stability  over  time. 

2.  No  impediments  to  the  movement  of  the  surgeon  or  the  patient. 

3.  If  a  position  localizer  is  used,  its  integration  into  the  surgical  environment  should  be  seam¬ 
less. 

4.  A  number  of  different  surgical  instruments  should  be  easily  integrated  with  the  system. 
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5.  Multiple  imaging  modalities  should  be  possible. 

6.  The  surgeon’s  direct  interactions  with  the  software  should  be  minimized. 

7.  The  interface  to  the  system  should  be  as  intuitive  as  possible. 

8.  A  means  for  comparing  the  results  obtained  via  image-guided  and  conventional 
approaches  should  be  possible  until  confidence  is  developed. 

9.  Redundancy  should  be  built-in;  failure  of  the  image-guided  component  should  not  require 
terminating  the  procedure. 

10.  Modularity  should  be  built  into  the  system  so  as  new  technologies  become  available  they 
can  be  easily  integrated. 

Most  of  the  enabling  technologies  such  as  medical  imaging,  sensing,  tracking  and  registration 
play  important  roles  in  the  context  of  image-guided  procedures.  It  is  expected  that  future 
developments  in  these  technologies  will  greatly  enhance  the  capabilities  of  image-guided  sys¬ 
tems.  Current  limitations  in  image-guided  systems  can  be  attributed  primarily  to  deficiencies 
in  these  underlying  technologies. 

Validation  and  critical  evaluation  of  any  image-guided  system  must  be  performed  before  it  can 
be  used  on  a  widespread  clinical  basis.  Validation  should  include  studies  regarding  possible 
failure  modes,  component  and  system  accuracies,  interface  and  usability  issues,  etc.  In  addi¬ 
tion,  clinical  evaluation  must  be  performed  to  demonstrate  that  the  image-guided  outcome 
warrants  the  expense  and  development  of  the  device. 

Some  examples  of  image-guided  procedures  which  have  been  submitted  in  the  pre-workshop 
papers  include:  the  ISG  Viewing  Wand  [14];  the  Grenoble  group’s  ACL  reconstruction  and 
grafting  system  [8];  StealthStation  for  pedicle  screw  insertion  and  other  procedures  [11];  and 
Edward’s  surgical  microscope  augmented  reality  system  [9]. 

1.3  Robotics  /  Manipulators 

In  the  context  of  the  RCAMI  workshop.  Robotics  /  Manipulators  will  include  the  intra-opera¬ 
tive  use  of  active  or  semi-active  robotic/manipulation  systems  to  assist  in  image-guided  surgi¬ 
cal  procedures.  The  terms  active  and  semi-active  are  defined  as  in  [5]: 

A  semi-active  system  is  one  in  which:  “the  action  is  physically  constrained  to 
follow  a  pre-defined  strategy.  The  action  is  guided  which  means  that  the  inter¬ 
vention  is  performed  with  respect  to  a  previously  defined  strategy,  but  its  final 
control  depends  on  the  surgeon”. 

An  active  system  is  one  in  which:  “some  subtasks  of  the  strategy  are  performed 
with  the  help  of  an  autonomous  robotic  system,  supervised  by  the  surgeon  and 
controlled  by  redundant  sensors”. 
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While  a  lot  of  research  has  been  performed  in  the  area  of  rehabilitation  and  medical  service 
robots  [10],  in  the  context  of  the  RCAMI  workshop,  the  robotics/manipulators  application 
area  will  be  limited  to  robotic  systems  that  are  used  during  the  course  of  a  medical  procedure. 

Recognized  advantages  of  robotic  systems  include:  the  ability  to  accurately  position  and  repo¬ 
sition  surgical  tools;  the  ability  to  apply  precisely  calibrated  forces;  the  potential  for  a  reduc¬ 
tion  in  tremor  as  compared  to  human  hands;  the  ability  to  scale  the  magnitude  of  forces  and 
motions  either  larger  or  smaller  than  those  possible  by  humans;  the  ability  to  provide  a  rigid 
platform  for  supporting  cameras  or  surgical  tools  in  a  tireless  manner. 

Recognized  limitations  of  robotic  systems  include:  current  commercially  available  robots  are 
intrinsically  unsafe  and  not  meant  for  direct  interaction  with  humans;  when  active  robotic  sys¬ 
tems  are  used,  the  surgeon  must  relinquish  a  certain  amount  of  control  to  the  robot  -  therefore 
the  ability  to  monitor  the  system’s  progress  is  paramount;  autonomous  robots  are  only  as  reli¬ 
able  as  the  strategy  and  software  which  controls  them  -  unexpected  conditions  can  be  prob¬ 
lematic;  mechanical  robotic  systems  may  require  frequent  calibration  to  ensure  accurate 
operation. 

Some  of  the  issues  in  this  application  area  include: 

1.  Should  robots  be  designed  on  a  proeedure-specific  basis,  or  is  it  possible  to  build  general 
purpose  medical  robots  which  satisfy  the  requirements  of  a  large  number  of  proeedures? 

2.  For  what  types  of  procedures  should  a  robot  be  used?  What  task  requirements  best  match 
the  strengths  of  robotic  systems?  Does  a  procedure  require  a  robotic  system  or  can  some 
other  approach  be  used? 

3.  How  can  the  safety  of  the  system  be  ensured?  Are  there  alternative  approaches  in  the  event 
of  failure  midway  through  a  robotic  procedure? 

4.  How  much  system  redundancy  is  required  and  how  should  it  be  incorporated  into  a 
design? 

5.  How  should  robotic  systems  be  designed  so  that  they  are  easily  controlled  and  monitored 
by  the  surgeon? 

The  Robotics  /  Manipulators  application  area  has  some  overlap  with  other  areas.  For  example, 
telesurgical  systems  may  require  robotic  hardware  very  similar  to  that  required  by  active  robot 
systems.  However,  requirements  for  such  systems  may  differ  in  terms  of  control,  sensing  and 
transduction  abilities,  status  monitoring,  etc.  Robotic  /  manipulation  systems  also  rely  heavily 
on  surgical  simulation  for  construction  of  pre-operative  plans. 

Some  example  robotic  systems  include:  RoboDoc  -  a  system  for  femoral  milling  in  total  hip 
replacement  surgery  [17];  Taylor’s  laparoscopic  camera  positioning  robot  [23];  Masamune’s 
MRI  needle  biopsy  system  [18];  Aesop  laparoscopic  camera  positioning  robot  [20];  Brett’s 
ear  drilling  system  [2];  and  Harris’  prostate  resectioning  robot  [12]. 
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1.4  Teleintervention 

In  the  context  of  the  RCAMI  workshop,  teleintervention  is  defined  as  the  use  of  remote  infor¬ 
mation  and/or  guidance  in  medical  interventions  involving  the  transduction  and  transmission 
of  perceptions  and/or  actions  at  a  distance. 

In  the  broadest  sense,  teleintervention  encompasses  everything  from  remote  surgery  with 
robotic  end  effectors  to  interactive  surgical  consultations,  teleconsulting,  and  remote  diagno¬ 
sis.  For  the  RCAMI  workshop,  the  term  teleintervention  applies  to  the  remote  use  of  robotic 
systems  (i.e.,  telerobotics),  interactive  surgical  consultation,  and  remote  patient  examination  / 
diagnosis.  It  does  not  include  non-real-time  teleconsulting  such  as  performed  by  some  radio¬ 
logical  reading  services.  Teleinterventional  applications  that  attempt  to  reduce  the  “distance” 
between  the  specialist  and  the  patient  while  making  these  interactions  as  seamless  as  possible 
are  the  primary  focus  of  the  teleinterventions  working  group. 

The  teleintervention  appUcation  area  has  a  large  overlap  with  the  robotics/manipulator  area. 
As  mentioned  above,  robotics  technologies  are  often  required  for  use  in  telesurgical  systems. 
Also,  teleinterventions  can  draw  upon  techniques  developed  for  image-guided  procedures  for 
tasks  such  as  guiding  a  remote  medic  through  a  surgical  procedure. 

Some  of  the  key  issues  to  be  addressed  in  the  teleintervention  application  area  include: 

1 .  Communication  and  device  bandwidth  requirements.  How  much  bandwidth  is  needed  to 
perform  a  particular  task? 

2.  Latency  -  How  much  latency  can  a  teleintervention  system  tolerate  before  becoming  unus¬ 
able?  How  can  latency  problems  be  reduced? 

3.  How  can  patient  confidentiality  be  ensured?. 

4.  How  can  teleinterventional  user  interfaces  be  designed  so  that  clinicians  feel  as  if  they  are 
present  at  the  remote  site? 

5.  What  technologies  are  required  to  allow  the  sensing  and  transduction  of  tactile  and  haptic 
sensations. 

Some  examples  of  teleinterventional  systems  include:  SRFs  remote  manipulator  system  [15] 
[1];  Moore’s  telementoring  laparoscopic  system  [19];  and  Howe’s  remote  palpation  work 
[13]. 
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Appendix  J  Workshop  Questionnaire 
Response  Summary 

This  section  of  the  report  has  been  compiled  by  grouping  the  responses  from  the  workshop 
participant  questionnaires.  The  contents  of  the  responses  presented  below  have  not  been  mod¬ 
ified  in  any  way.  Among  the  questionnaire  respondents,  the  average  number  of  years  working 
in  the  RCAMI  field  is:  8.2.  The  breakdown  of  primary  area  of  interest  among  respondents  is: 


Image-guided  procedures 

22 

Robotics  /  Manipulators 

10 

Surgical  Simulators 

15 

Teleinterventions 

6 

J.l  RCAMI  Common  Issues  and  Technical  Problems 

What  are  the  major  technical  problems  and  research  needs  in  RCAMI? 

Imaging  technology  (displays);  hardware  (manipulators,  actuators);  communications  (real¬ 
time  codecs);  networking;  FDA  and  equivalents;  well-defined  applications;  understanding  of 
cognitive/psychomotor  skills  and  skill  acquisition  in  surgery;  lack  of  quantitative  metrics; 
aggressive  testimonial  presentations;  legal  problems;  safety  measures;  time  delay;  surgeon- 
machine  interface;  precision  of  telemanipulators;  image  quality;  lack  of  funding;  limited  phy¬ 
sician  understanding  of  technical  possibilities;  lack  of  engineering  understanding  of  physician 
patient  needs;  need  for  decreasing  cost  of  technology;  need  for  robotic  devices  capable  of  tis¬ 
sue  handling;  post  graduate  education;  need  for  real  time  high  resolution  video  on  image 
transfer;  technical  problems  associated  with  the  challenge  of  cost  reduction;  sterilizability; 
cleanability;  reliability;  maintainability;  compatibility  with  existing  surgical  equipment; 
awareness  of  alternative  technical  (&  pharmaceutical)  approaches  that  would  abbreviate 
RCAMI;  assessment  of  value  added  with  each  increase  in  complexity;  selection  of  economi¬ 
cally  viable  clinical  applications;  inadequate  realism  and  breadth  and  richness  of  the  experi¬ 
ence  in  surgical  simulation  in  order  to  develop  more  effective  interfaces  for  telemanipulation; 
improvement  needed  to  stereographic  visualization;  resolution  and  dynamic  range  improve¬ 
ment  needed  in  head-mounted  displays;  mechanical  design  of  small  multi-dof  manipulators 
with  high  stiffness,  low  friction  &  backlash  (good  reliability);  understanding  of  what  surgeon 
do  in  various  procedures  (which  actions  are  essential,  which  are  not;  what  sources  of  informa¬ 
tion  are  used,  which  ignored,  etc.  coordination  of  visual  and  kinesthetic  motion;  registration 
of  3-D  imaging  with  mobile  tissues;  ignorance  —  engineers  of  medicine  -  physicians  of  tech¬ 
nology;  suturing;  palpation;  cooperative  manipulation  —  robotic  assistance  to  surgeons;  mini¬ 
mizing  trauma  to  tissue  during  grasping  &  manipulation;  lack  of  measurement  of  mechanical 
properties  of  tissues;  fundamental  relationship  between  task  description  and  dexterity  require- 
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ments;  variety  of  opinions  among  surgeons  (often  contradictory!);  fully  automated  segmenta¬ 
tion;  real  time  see  through  displays;  methods  for  visually  tracking  position  of  scopes  relative 
to  3D  scans;  realistic  simulations  (i.e.,  haptics  &  visual);  effective  in-theatre  registration  and 
tracking;  effective  3D  monitoring  of  patient  in  real  time;  fusion  of  real  time  and  pre-operative 
sensor  information;  clearer  definition  of  clinical  requirements;  sufficient  dexterity  for  surgi¬ 
cal  task;  solution  to  problem  of  need  for  real-time  response;  standards  lacking;  metal  artifact 
(CT&MR);  real  time  3D  imaging;  generalizability  of  results;  stereolithography  is  too  expen¬ 
sive;  motion  blurring;  target  definition  requires  multimodality;  MRT  experience  is  limited;  CT 
fluoroscopy;  lack  of  concepts;  tool  development;  acceptance  by  MDs;  integration  imaging 
technology  (accuracy,  speed,  size,  cost,  existence  of  invisible  tissue);  image  processing 
method  (analysis  -  [segmentation,  etc.]  recognition  [understanding],  display  [graphics,  etc.]); 
sensing  device  like  fingers  and  hands  of  human;  man-machine  interface  (mechanical  I/O, 
audio  and  visual)  models  and  physics  of  human  tissue  (skin,  muscle,  bone...)  computers  and 
processors  (size,  weight,  robustness,  operability,  fail-safe  system)  mechanics  (way  to  manipu¬ 
late  living  organs  by  machine);  basic  data/model  collection  and  establishment;  normal  data¬ 
base  establishment;  conduct  control  studies;  cost  effectiveness;  availability  of  information  in  a 
timely  basis;  reliability;  depth  perception;  clinical  lag;  new  skill  requirements;  lack  of  out¬ 
come  evidence;  stereotactic  errors;  lesions  localization;  medico-legal  anxieties;  training  of 
staff;  clinical  requirements;  apathy;  space  of  the  operative  field;  duration  of  RCAMI-specific 
tasks;  lack  of  infrastructure;  need  for  skilled  personnel;  quality  of  conventional  procedures; 
lack  of  clinical  collaborators  that  are  willing  to  truly  immerse  themselves  in  these  problems; 
lack  of  technical  people  willing  to  truly  understand  clinical  needs;  standardized  test  data  sets 
for  comparing  algorithms/reporting  results;  making  successful  software  generally  available 
for  use  by  the  community  (must  permit  integration  of  future  developments);  patient  accep¬ 
tance  levels;  device  reliability;  inadequate  clinician  training;  inadequate  peripheral  monitor¬ 
ing;  insufficient  design  guidelines;  sensors  for  touch;  distributive  sensing  and  actuation 
techniques;  confidence  in  the  business  community;  risk  assessment  methods;  operating  set  up 
times;  guarding  intellectual  property;  difficulty  with  institutional  review  boards;  strapped 
financial  situation  of  US  medical  schools;  desire  in  others  to  “reinvent  the  wheel”;  demands 
on  educating  others  once  a  system  has  been  approved  for  general  use;  lack  of  standards 
between  imaging  technologies  for  file  transfer;  physician  confidence;  perception  it  is  “costly” 
affordable  systems;  end  effector  “tool”  development;  anatomical  based  simulations;  patient 
motion-image  update;  real-time  intra-operative  imaging  and  image  reconstruction;  legal  rami¬ 
fications  of  telesurgery;  compactness  of  mechanisms,  segmentation;  commonality  of  soft¬ 
ware;  non-technical  users;  pre  &  intra-operative  planning;  augmented  benefit  from  computer 
assisted  planning  (by  FEM-analysis,  atlases);  No  pressure  on  NHS  hospital  to  pursue  these 
sort  of  activities;  Lack  of  support  from  manufactures  -  particularly  in  the  UK;  too  early  com¬ 
mercialization  in  MRCAS;  dexterous  manipulator;  security  of  network;  accuracy  of  data 
globe;  sectionalism  among  hospital  departments  (esp.  radiology),  who  pays  the  cost?;  medical 
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insurance  system;  lack  of  funding  (conflict  between  NSF  (no  medical)  and  NIH  (no  techni¬ 
cal)). 

Table  J-1,  which  spans  several  pages,  further  develops  nine  of  the  most  commonly  addressed 
technical  issues  and  problems. 

What  areas  in  RCAMI  are  currently  NOT  receiving  enough  attention? 

Fundamental  human  performance/human  computer  interface  issues;  micro  technology 
(MEMS);  legal  problems;  pathology  affecting  soft  tissues  (i.e.,  chest,  abdomen);  sensory  feed¬ 
back  (force,  touch,  vision);  methods  for  careful  validation  of  segmentation  &  registration; 
safety;  integration  of  real  time  sensor  information  with  pre-op  information  (sensor  fusion  and 
augmented  reality);  lack  of  training;  planning  and  guidance  systems;  evaluation  -  by  rigorous 
methodology  (RCT);  generalizability  of  results;  low  cost  systems;  image  guidance;  therapy 
methods;  clinical  trials;  basic  model  and  data  development;  reliability  study;  robustness  of 
computer  networks;  endoscopy;  system  engineering  for  applications  marketing;  algorithms 
and  implementations;  latency;  development  of  a  completely  reliable  imaging  file  organization; 
integration  of  head  mounted  displays  into  image  guided  surgery;  integration  of  micro  robots 
into  image  guided  surgery;  development  of  surgical  robots  for  use  in  amenable  surgery,  e.g., 
orthopaedics;  development  of  registration  devices  to  guide  surgery;  inexpensive  image  guid¬ 
ance  systems  for  image  guided  therapy;  infrastructure  &  standards;  alternative  solution  (non- 
RCAMI)  for  clinical  problems  in  the  UK;  manipulatory  robotic;  formation  and  coordination 
of  international  databases;  development  and  distribution  of  normative  data;  methods  for  care¬ 
ful  validation  of  segmentation  &  registration;  realistic  haptic/visual  integrated  simulators. 

What  piece  of  technology  do  you  wish  that  you  had  today? 

A  robotic  device  for  tissue  manipulation;  tactile  shape  display  with  high  force,  high  resolu¬ 
tion,  high  bandwidth;  combination  of  functional  mapping  and  CAS;  cheap  intra-operative  3D 
image  and  sensing  system;  cone  beam  CT  &  3D  fluoroscopy;  automatic  volume  image  seg¬ 
mentation  and  labeling;  real-time  image  processing;  3D  rendering;  very  accurate  imaging 
technology  such  as  recording  details  of  soft  bone,  soft  tissue,  and  of  3D  images  of  moving 
heart;  real-time  distance  imaging  with  tactile  feedback;  a  “good”  robot;  integration  of  CT, 
MRI,  PET,  Ultrasound,  Atlas  information;  a  reliable  free-hand  endoscopy  system  for  endo¬ 
nasal  surgery;  a  digital  operating  microscope  with  integrated  CAS-system;  real  time  MR 
image  acquisition/3D  volumetric  reconstruction  of  anatomical  objects;  100  Gbs/networks 
lower  latency;  surface  actuator  methods;  lightweight  stereoscopic  head  mounted  display; 
computer  based  registration  devices  in  the  operating  room;  real-time  data  feedback;  anatomi¬ 
cal  based  simulator;  automatic  registration  of  real  time  images  to  pre-op  scans;  compact,  light¬ 
weight,  accurate  3D  imaging  device;  non-intrusive,  passive,  non-line-of-sight  6  Dof 
positioning  device  accurate  to  ~0.1  mm/~0.03°;  optical  localization  system  (optotrak)  too 
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expensive;  surgical  simulator  telesurgery  system;  self  contained  6  dof  micro-localizer,  mount- 
able  on  a  tool  tip;  non-contact  non-wired  low-size  component  line  of  sight  independent  navi¬ 
gator. 

What  are  some  of  the  key  hurdles  to  overcome  in  sensing? 

3-D  Reconstruction/display  of  sensory  data;  tactile  sensing  in  a  realistic  way  is  still  far  away; 
endoscopes  with  improved  dexterity  and  intuitive  control  interface;  tracking  mobile  tissues 
(organs,  joint,  etc.);  wireless  and  accurate;  price;  integration  of  video  with  other  technologies 
esp.  ultrasound;  low  dose  x-ray;  development  of  versatile  3D  ultrasound;  sensing  the  tip  of 
flexible  tools  (catheter,  endoscope);  improved  intra-op  imaging;  non-invasive  measurements; 
radiation;  size  of  sensors/devices/modelling;  detectors;  displays  and  perception;  making  sys¬ 
tems  less  sensitive  to  working  environment. 

What  are  some  of  the  key  hurdles  to  overcome  in  execution? 

Micro  instrumentation;  simpler  designs;  lower  cost  motors;  inherent  reliability  (hardware  & 
software)  in  the  hands  of  ordinary  medical  support  staff  who  have  little  time  -  all  these  would 
help;  ergonomics  of  interfaces;  coordination  of  kinesthetic  control  &  visual  feedback;  3D  real 
time  visualization  system  with  contrast  mechanism  to  update  as  procedure/intervention  pro¬ 
ceeds;  non-invasive  measurements;  efficient  ways  to  code  complex  tasks  for  distributed  pro¬ 
cessing;  sterilization  of  key  devices  and  sensors  to  invasive  use;  registration  and  image 
rendering  speed 

What  are  some  of  the  key  hurdles  to  overcome  in  planning? 

3-D  data  sets;  rapid  conversion  of  imaging  data  to  the  form  needed  for  planning;  use  of  the 
diagnostic  imaging  modality  required  for  standard  work  up,  without  having  to  do  extra  scans 
just  to  satisfy  needs  of  the  computer-assisted  planner;  deformation;  integrating  fast  but  cost- 
effective  computing  systems  with  3D  voxel  models  derived  from  multiple  sources  for  effective 
planning  tools;  automatic  segmentation  and  labeling;  real  time  image  processing;  quality  of 
original  images;  I/O  devices  in  the  simulation;  sub  micron  models;  patient  acceptance;  better 
models;  lesion  segmentation;  registration;  display  mapping;  human  interfaces. 


What  will  be  possible  over  the  next  25  years? 

Numerous  endoluminal  (endovascular,  GI,  neurosurgery);  focusing  destroying  energies  for 
non-invasive  tumor  treatment;  simulation  of  surgical  approaches  for  non-fixed  organs,  robotic 
assistance  for  surgery  of  these  organs;  incisionless  surgery  using  transcutaneous  technology 
such  as  high  intensity  focused  ultrasound;  3D  sensor  should  be  more  easy-handled;  brain  shift 
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during  the  surgery  should  be  overcome;  extending  minimally  invasive  image  guided  proce¬ 
dures  to  other  parts  of  the  body  esp.  abdomen  and  pelvis;  3D  Rad  Tx;  minimally  invasive  Tx; 
CABG;  bowel  anastomosis;  node  dissection;  image  guided  endoscopy  MR  guided  FUS; 
recording  images  augmented  by  much  more  details  of  organs  and  tissues  in  human  body;  dis¬ 
tance  procedures;  microsurgery;  surgery  within  confined  spaces;  cochlear  surgery  for  tinnitus; 
retinal  implant;  model-based  functional  septum  surgery;  less  invasive  heart  bypass  surgery;  a 
host  of  laparoscopic  procedures;  telerobotic  brain  transplantation;  extreme  minimally  invasive 
surgery  using  micro-robotics  into  critical  areas  of  the  brain;  further  shrink  the  surgical  field 
into  the  microscopic  range;  complex  orthopaedic  alignment  procedures;  soft  tissue  reposition¬ 
ing  procedures;  tumor  excision  procedures;  customized  preparation/insertion  of  body  parts/ 
substitutes;  various  forms  of  conformal  localized  therapy;  contact  less  percutaneous  localiza¬ 
tion. 


J.2  Image  Guided  Procedures 

On  a  scale  from  1  -  5  (5  being  very  difficult)  how  difficult  has  it  been  to  convince  medical 
colleagues,  research  colleagues  and  patients  (if  applicable)  to  support  image  guided  proce¬ 
dures? 

Medical  Colleagues: 


1 


■f - ^ ^ - 1 

2  3  4  5 

Medical  Colleagues:  2.02 


What  are  the  main  areas  of  resistance  from  medical  colleagues? 

Accuracy;  reliability;  safety;  clinical  relevance;  cost  effectiveness;  benefits  are  obvious; 
understanding  concept;  suspicion  of  “high  tech”  procedures  and  their  cost;  as  computers  have 
become  used  more  widely  in  medicine  -  physicians  are  more  willing  to  accept  practicability; 
PACS/IMAG  systems;  what  real  improvements  are  provided  (e.g.,  conformal  3d  treatment 
planning);  too  complicated  takes  too  much  time;  use  of  expensive  imaging  technologies  (e.g. 
CT  Scans)  as  basis  for  computer  assisted  surgery  procedures;  cost  of  hardware;  most  realize 
computers  can  provide  considerable  assistance;  need  to  show  concrete  definite  advantage; 
time  stress  in  clinical  routine;  need  to  registration  of  systems  with  patient  to  sub-millimeter 
level;  It’s  a  new  concept  that  requires  repeated  review  so  that  clinical  colleagues  begin  to  see 
application  in  their  field;  learning  time;  non  user-friendly  systems;  Ego  problems;  missing 
educational  background. 
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Research  Colleagues: 


1  2  3  4  5 

Research  Colleagues:  2.10 

What  are  the  main  areas  of  resistance  from  research  colleagues? 

Technical  feasibility;  cost  effectiveness;  theoretical/research  problems  are  few  —  haven’t  the 
interesting  issues  been  addressed;  lack  of  convincing  scientific  database;  preference  for 
robotic  systems;  defining  a  real  scientific  problems;  the  only  obstacle  was  the  desire  to  protect 
one’s  intellectual  property;  funding;  Reliability  &  Cost;  absence  of  standards  or  accepted 
leader;  no  main  obstacles. 

Patients: 


1  2  3  4  5 

Patients:  1.50 

What  are  the  main  areas  of  resistance  from  patients? 

How  accurate  is  it?  safety/morbidity;  patients  have  been  very  receptive  towards  this  technol¬ 
ogy;  they  understand  this  better  and  are  less  skeptical;  concerned  with  reliability. 

100%  of  the  respondents  said  image  guided  procedures  would  be  widely  used  in  the  next  5- 
10  years. 

What  applications  will  image  guided  procedures  be  widely  use  for  in  the  next  decade? 

When  will  they  be  introduced? 

Surgical  guidance  1-3  years;  targeting  of  parenchimal  space  occupying  lesions  3  years;  joint 
surgery  1-2  years;  biopsy  of  intra  abdominal  organs  5  years;  radiation  treatment,  minimally 
invasive  scope  surgery,  neurosurgery,  biopsy,  international  radiology  &  M.I.T.  1-3  years; 
MRT:  focused  US,  CRYORX,  Laser  IS  2  years;  CT  fluoroscopy  -  cone  beam  &  area  detectors 
3  years;  frameless  stereotaxy  3  years;  surgery  under  the  guidance  of  CT  1-2  years;  surgery 
under  the  guidance  of  MRI. 


105 


Appendix  J  -  Workshop  Questionnaire  Response  Summary 


If  you  think  Image-guided  procedures  will  not  be  widely  used  over  the  next  decade,  why 
not? 

Cost 

40%  of  the  physicians  are  currently  using  image  guided  procedures. 

If  you  are  a  physician  and  don’t  use  image  guided  procedures,  why  not? 

Not  available;  down  sizing  of  computer;  easy  handling  of  the  system;  financial  (insurance) 
support  of  it’s  use;  cost 

What  are  the  top  technical  problems  with  image  guided  procedures? 

Image  registration;  real  time  updates;  use  in  soft  tissues;  reliability;  speed;  none;  computer 
vision  improvements;  on-line  segmentation;  medical  image  improvements  (x-ray...);  ease  of 
use  including  augmented  reality  (microscopes  &  endoscopes);  validation  and  improved  accu¬ 
racy  within  a  procedure;  MR  Interventional  Systems  (Open  MR,  MRT);  image  volume  fusion 
(electronic  atlas);  cone  beam  CT  scanner;  faster  image  processing;  better  imaging;  integration 
with  therapy  devices;  outcome  study;  fidelity;  registration/fusion  of  multimodal  information; 
better  tracking  technology;  formalized  safety  assessment  methods;  methods  to  assure  software 
integrity;  integration  of  pre-op  planning  with  surgical  procedures;  cost  of  CT  scans;  miniatur¬ 
ization  of  sensors. 

What  are  the  top  regulatory  problems  with  image  guided  procedures? 

Reimbursement;  reduced  cost;  access  to  image  and  other  information  that  really  works;  regu¬ 
latory  framework  that  does  not  inhibit  innovation;  proof  of  benefit  -  by  rigorous  methods 
(RCT);  medical  acceptance;  increased  acceptance  of  effectiveness  &  importance;  medico¬ 
legal;  clinical  lagapathy;  appropriate  evaluation  and  testing;  training  and  regulations  for  use; 
methods  of  sharing  risk  &  liability  between  professional  bodies;  confidence  in  the  business 
sector;  education  of  physicians  as  to  benefits  of  FDA  approved  devices;  insurance  company 
reluctance  to  support  investigational  studies;  cost  of  FDA  submission;  published  policy  for 
FDA. 
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J.3  Surgical  Simulators 

On  a  scale  from  1  -  5  (5  being  very  difficult)  how  difficult  has  it  been  to  convince  medical 
colleagues,  research  colleagues  and  patients  (if  applicable)  to  support  surgical  simulators? 

Medical  Colleagues: 


I - •\ - ^ ^ - 1 

1  2  3  4  5 

Medical  Colleagues:  1.95 

WhcU  are  the  main  areas  of  resistance  from  medical  colleagues? 

Lack  of  useful  prototypes;  reliability  and  ease  of  use;  relevant  information;  cost/availability; 
complex  to  use  (requires  expertise  in  computer  graphics,  too  slow);  time  required;  not  being 
able  to  incorporate  all  clinical  situations  in  the  planning;  suspicion  of  “high  tech”  procedures; 
lack  of  training  programs;  time  many  colleagues  not  planning  the  interventions  formally; 
insuring  that  the  simulation  (even  when  integrated  with  data)  is  as  real  as  possible  and  is  not 
“creating”  impossible  situations;  realistic  methods  to  enable  touch  sensation;  computation/ 
speeds;  lack  of  perceived  value  over  current  planning  methods;  lack  of  faith  in  simulator  really 
providing  realistic  environment  for  training  and  pretreatment  planning;  show  definite  advan¬ 
tages;  ability  to  execute  plans;  additional  procedures  induce  additional  work  load  in  the  first 
step;  conventional  planning  is  very  time-effective;  “Good  idea  but  how  does  it  apply  to  me?”; 
running  the  software  on  inexpensive  platforms;  the  development  of  detailed  tests  of  the  simu¬ 
lators;  Pre-op  planning  takes  “too  much  time”  for  some  clinical  colleagues;  obstinance:  “I 
don’t  need  this”;  fear  of  computers;  lack  of  intent/cooperation  from  data  acquisition  person¬ 
nel. 

Research  Colleagues: 


1 


3 


4 


5 


Research  Colleagues:  1.81 
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What  are  the  main  areas  of  resistance  from  research  colleagues? 

Introduction  of  clinical  principles;  lack  of  clinical  expertise;  accuracy  -  validity;  repeatability  - 
precision;  cost;  not  basic  science  enough;  reluctance  to  use  standard  software;  more  realistic 
models  vs.  assembling  all  the  needed  parameters  and  computation  time;  skeptical  that  medical 
community  will  not  accept  them  if  they  really  don’t  simulate  actual  clinical  environment; 
funding;  adequacy  with  needs;  imaging  problems;  bringing  together  researchers  and  clinicians 
to  determine  goals;  high  cost  of  computer  platforms;  expertise  required  to  run  application  pro¬ 
grams;  reluctance  to  leave  known  methods;  absence  of  normative  database;  absence  of  analy¬ 
sis  conventions. 

Patients: 


1  2  3  4  5 

Patients:  1.69 

What  are  the  main  areas  of  resistance  from  patients? 

Sometimes  used  in  cases  where  benefits  are  dubious  (e.g.,  cleft  palate);  How  to  carry  out  the 
precision  provided  by  the  computer  model  analysis;  cost  for  the  patient;  no  problem;  patients 
are  very  receptive;  none;  families  and  older  patients  recognize  the  skill  icon  and  I  finally  get 
consent. 

95%  of  the  respondents  said  surgical  simulators  would  be  widely  used  in  the  next  5-10 
years. 

What  applications  will  Surgical  Simulators  be  widely  use  for  in  the  next  decade?  When  will 
they  be  introduced? 

Basic  skill  practice  5  years;  rehearsal  of  uncommon  procedures  8  years;  image  augmentation 
8  years;  train  medical  support  specialists  2-3  years;  sinus/head  &  neck  surgery/cranio-facial 
(endosurgery,  especially)  3-4  years;  dexterity  training  &  measurement  for  minimally  invasive 
surgery  (laparoscopy,  endoscopy,  etc.)  2-4  years;  3D  surgical  maneuver  simulation  3  years; 
skull  base  surgery  1-2  years;  neurosurgery  1-2  years;  some  forms  of  radiotherapy  ~3  years; 
urology  Syears;  determine  accuracy  of  procedures  2-4  years;  integrate  function  of  surgical 
teams  3-6  years;  gesture  training  for  Laparoscopy  2  years. 
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If  you  think  surgical  simulators  will  not  be  widely  used  over  the  next  decade,  why  not? 

Not  available  in  most  specialties;  lack  of  authenticity  and  complexity  may  limit  usefulness  of 
trainers  in  laparoscopic  surgery;  lack  of  fidelity;  the  whole  concept  of  surgical  planning  does 
not  seem  consistent  with  most  surgeon’s  personalities;  most  surgeons  will  not  take  the  time  to 
do  planning  prior  to  a  surgery;  what  is  needed  is  a  device  that  seamlessly  integrates  with  the 
surgical  act  that  allows  the  surgeon  to  do  planning  on  the  fly;  no  proof;  reasonable  cost/benefit 
ratio 

24%  of  the  physicians  are  currently  using  surgical  simulators. 

If  you  are  a  physician  and  don’t  use  a  surgical  simulator,  why  not? 

No  useful  system;  not  available  in  my  specialty;  cost;  we  have  intra-operative  surgical  local¬ 
ization  devices  that  allow  us  to  plan  surgery  on  the  fly  intra-operatively;  Systems  unavailable 
due  to  simplicity 

What  are  the  top  technical  problems  with  surgical  simulators? 

Photo-realistic  graphics  or  equivalent;  deformability/fluid  motion;  haptic  interface;  sensors; 
computer  graphic  refinement;  availability;  improved  haptic  interfaces;  robust  systems;  realis¬ 
tic  nonlinear  tissue  behavior;  none;  down  sizing  of  computer;  digital  transfer  to  the  computer; 
realistic  display;  ease  of  use;  ergonomically  related  to  surgical  procedure;  proof  of  benefits; 
registration  to  patient;  enhanced  reality;  performance  of  image  processing;  improvement  of 
image  quality;  reliability;  improved  automated  &  semi-automated  segmentation;  more  inter¬ 
disciplinary  research  activity  reliable  stereoscopic  head  mounted  display;  high  resolution  pho¬ 
tographic-type  quality  computer  displays;  planners  must  be  automated;  real  time  interactive 
response;  ability  to  carry  out  planned  intervention;  infrastructure;  multi-application  usability; 
reliable  and  automatic  reconstruction;  biomechanical  and  physiological  simulation. 


What  are  the  top  regulatory  problems  with  surgical  simulators? 

Surgery  as  a  skill  (vs.  an  art);  quantitative  assessment  of  performance  liability;  safety;  confi¬ 
dentiality;  change  in  resident  training  program  philosophy;  cost-effectiveness;  recognition  of 
the  need  for  specialized  training  (lecture,  laboratories,  exercises,  etc.);  none;  daily  and  routine 
use  of  computer  by  doctors;  standards  of  information  exchange  that  really  work;  ensuring  reg¬ 
ulatory  changes  do  not  stifle  innovation;  proof  that  systems  are  safe  and  efficacious;  clinical 
measures;  increased  acceptanee  of  effectiveness  &  importance;  more  data  on  cost-benefit  to 
persuade  purchasers;  interstate  license/regulation;  medico-legal;  clinical  lagapathy;  surgeons 
have  to  accept  the  use  of  off-line  training;  general  adoption  of  computer  savvy  on  the  part  of 
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surgeons;  increasing  demands  for  surgeons  to  perform  continuing  medical  education;  routine 
presence  of  computers  in  operating  suites;  medical  insurance;  engineers  must  apply  solution 
not  technology;  get  acceptance  that  simulators  can  be  used  for  accreditation;  clarify  FDA  pol¬ 
icy  on  software. 

J.4  Robotics  /  Manipulators 

On  a  scale  from  1  -  5  (5  being  very  difficult)  how  difficult  has  it  been  to  convince  medical 
colleagues,  research  colleagues  and  patients  (if  applicable)  to  support  robotics  /  manipula¬ 
tors? 

Medical  colleagues: 


1  2  3  4  5 

Medical  Colleagues:  3.25 

What  are  the  main  areas  of  resistance  from  medical  colleagues? 

Fear  of  loss  of  control;  Lack  of  data  demonstrating  equivalent  outcomes  and  benefits;  small 
changes  in  current  procedures  (additional  information  sources,  improved  instruments)  are 
easy;  major  changes  are  difficult;  fear  of  complexity;  cost  justification;  lack  of  space  in  OR; 
reluctance  to  admit  that  current  procedures  are  difficult;  credibility;  robustness;  reliability;  rel¬ 
evance;  lack  of  proof  for  benefits;  most  haven’t  thought  about  how  to  use  it;  lack  of  effective 
PAGS/IMAC  systems  liability  or  time  consumption;  lack  of  an  understanding  of  what  can  be 
done;  surgeons  are  very  concerned  about  the  safety  of  this  technology  and  the  possible  impact 
it  will  have  on  their  financial  bottom  line;  awkwardness  of  current  technology;  lack  of  avail¬ 
ability  of  user  friendly  systems;  developing  systems  to  fit  individual  surgical  environments 
and  needs;  comparative  advantage;  efficiency;  Colleagues  think  the  process  too  complicated; 
impractical;  too  far  in  future;  people  are  resistant  to  robots  due  to  registration  errors;  FDA 
approval;  learning  curve;  lack  of  demonstrated  clinical/cost  benefits;  science  fiction  image  of 
robots. 
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Research  colleagues: 


1  2  3  4  5 

Research  Colleagues:  2.46 

What  are  the  main  areas  of  resistance  from  research  colleagues? 

Generally  eager  to  get  involved;  technical  feasibility;  doubtful  academic  value;  “solution  in 
search  of  a  problem;  need  to  build  up  the  basic  data;  cost;  an  assurance  that  all  reasonable 
design  considerations  have  been  made;  only  recently  in  Europe  are  standards  being  devel¬ 
oped;  it  has  been  extremely  difficult  dealing  with  the  politics  between  companies  and  the 
appropriate  desire  to  protect  one’s  intellectual  property;  technical  problems  and  fail  safe  clini¬ 
cal  requirements  -  big  tasks  -  not  easy  to  approach  and  overcome;  funding  in  medical  robotics 
has  been  limited;  transition  from  lab  prototype  to  real  product  seems  unclear;  coemption  from 
optical  tracking  systems. 

Patients: 


1  2  3  4  5 

Patients:  2.35 

What  are  the  main  areas  of  resistance  from  patients? 

Patient’s  are  very  willing  and  exciting;  high  cost;  low  availability;  loss  of  personal  contact 
with  physician;  there  have  note  been  any  medical  robotic  devices  available  for  clinical  use;  or 
even  research  use  in  neurosurgery;  will  it  work  for  me;  patients  want  robotics  to  assist  only, 
not  to  do;  in  UK  patients  trust,  science  fiction  image  of  robots  concerns  about  safety. 

Due  to  an  error,  the  following  questions  were  never  asked  on  the  questionnaire.  We  are  sorry 
any  inconvenience  or  problems  this  may  introduce. 


Ill 


Appendix  J  -  Workshop  Questionnaire  Response  Summary 


What  applications  will  robotics  /  manipulators  be  widely  use  for  in  the  next  decade?  When 
will  they  be  introduced? 


If  you  don’t  think  robotics  /  manipulators  will  be  widely  used  over  the  next  decade,  why 
not? 


If  you  are  a  physician  and  don’t  use  robotics/manipulators,  why  not? 

What  are  the  top  technical  problems  with  robots  /  manipulators? 

What  are  the  top  regulatory  problems  with  robots  /  manipulators? 

J.5  Teleinterventions 

On  a  scale  from  I  -  5  (5  being  very  difficult)  how  difficult  has  it  been  to  convince  medical 
colleagues,  research  colleagues  and  patients  (if  applicable)  to  support  teleinterventions? 

Medical  colleagues: 


1  2  3  4  5 

Medical  Colleagues:  3.20 

What  are  the  main  areas  of  resistance  from  medical  colleagues? 

Novel  technology;  no  clinical  trials;  fear  of  lack  of  redundancy  in  case  of  system  break  down; 
lack  of  data;  medicolegal  issues;  cost;  anticipation  that  collective  small  improvements  to  more 
conventional  methods  would  obviate  teleoperation’s  advantages;  scepticism  —  haven’t  seen 
teleop.  systems  that  work  as  well  as  direct  manipulation;  loss  of  control,  esp.  in  emergencies; 
relevance;  technical  feasibility;  uncertain  need  for  technology,  especially  in  urban  hospitals; 
data/image  transmission;  lack  of  training  programs;  no  network;  complex;  restricted  by  band¬ 
width;  over  enthusiasm/expectations;  demonstrating  usefulness;  ethical/legal  concerns. 
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Research  colleagues: 


I  ^  9  ^  I 

1  2  3  4  5 

Research  Colleagues:  2.95 

What  are  the  main  areas  of  resistance  from  research  colleagues? 

none  listed 

Patients: 


1 


2 


4 


Patients:  3.17 


5 


What  are  the  main  areas  of  resistance  from  patients? 

Concern  that  primary  physician  is  not  in  the  O.R.  or  that  someone  unknown  is  doing  surgery; 
is  it  safe?  any  other  way  to  do  this;  loss  of  personal  contact  with  physician;  easy  for  telediag¬ 
nosis;  telesurgery  remains  experimental 

51%  of  the  respondents  said  teleinterventions  would  be  widely  used  in  the  next  5-10  years. 

What  applications  will  teleinterventions  be  widely  use  for  in  the  next  decade?  When  will 
they  be  introduced? 

Minimal  access  surgery  3  years;  micro/endo  surgery  5  years;  remote  surgery  8  years;  interven¬ 
tional  diagnostic  procedures  in  contaminated  areas;  space  missions,  ambulances  5  years; 
robotization  of  surgical  tools  2  years;  military  -  triage  10  years;  microsurgery  (opthamology, 
neurosurgery)  over  short  distance  (same  room)  5-10  years;  too  many  major  technical  prob¬ 
lems  to  overcome  in  that  time  period,  far  apart  islands  4-5  years;  rural/remote  emergencies  3 
years;  3rd  world/military/warfare  2  years;  super-specialization  5  years;  teleconsultation  3-4 
years;  remote  instructional  courses  5-10  years;  telementoring  will  increase  5-10  years;  biopsy 
apparatus  1-2  years;  removal  of  foreign  objects  in  airway  and  upper  GI  tract  5yrs 
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If  you  think  that  teleinterventions  will  not  be  widely  used  over  the  next  decade,  why  not? 

Lack  of  communication  infrastructure;  social  resistance;  initial  cost;  there  are  not  enough 
obvious  potential  applications  that  are  not  more  cost-effectively  handled  by  1)  on-site  semi¬ 
skilled  and  mentored  staff  or  2)  transport  to  a  medical  center;  patient  through-put  must  be 
high  to  justify  expensive  equipment;  surplus  of  surgeons;  surgeons  should  be  sufficiently 
trained;  reliability  of  computer  and  communication  systems  is  not  enough  in  emergency;  the 
advantages  are  not  so  great  that  wide  use  will  occur  within  10  years;  regulatory  requirements; 
regulation  bodies  will  require  considerable  reassurance  of  the  reliability;  too  risky  and  techno¬ 
logically  too  complex;  In  most  places  local  surgeons  would  be  available  with  sufficient  experi¬ 
ence;  development  of  reliable  systems  need  more  time;  clinical  added  value  often  unclear. 

14%  of  the  physicians  are  currently  using  teleinterventions. 

If  you  are  a  physician  and  don’t  use  teleinterventions,  why  not? 

No  need  presently  for  telesurgery;  not  in  the  philosophy  of  my  group;  doubtful  value  in  an 
academic  medical  center/urban  setting;  not  available  yet;  lack  of  national  “new”  ambulance 
service;  no  facilities  available. 


What  are  the  top  technical  problems  with  teleinterventions? 

Enhanced  displays;  increased  DOF;  improved  telecommunication  networks  regionally  and 
globally;  robotic  end  effector  development;  lower  cost  technology;  dexterity;  flexible  visual¬ 
ization  with  variable  point  of  view;  safety;  robustness;  speed;  immersion  in  realistic  environ¬ 
ment;  compact;  easy-to-use;  surgical  implements  &  appliances;  visual  &  other  forms  of 
feedback  surgeons  are  completely  comfortable  with;  reliability  of  systems;  micro-sensors  and 
actuators;  head  mounted  stereoscopic  displays;  integration  with  other  information;  bio  com¬ 
patibility;  materials;  on  site  skilled  surgeon. 

What  are  the  top  regulatory  problems  with  teleinterventions? 

Demonstration  of  cost  effectiveness;  acceptance  of  efficacy  by  medical  groups;  resolution  of 
medicolegal  issues;  defined  reimbursements/costs;  licensing  (state  to  state);  reduced  capital 
cost  of  equipment;  on-site  staff  skilled  to  intervene  surgically  and  to  provide  post-op  care; 
limited  medical  license  to  practice  (across  state  lines);  perceived  threat  to  local  practitioners; 
equipment  must  past  FDA  test  (IND,  510K,  PMA...);  medico-legal;  confidence  in  the  business 
community;  legal  concerns  regarding  malpractice. 
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Appendix  K  Working  Gronp  Issue/Topic 
Checklist 


This  document  contains  a  checklist  of  topics  and  issues  which  each  group  should  address  dur¬ 
ing  the  working  group  sessions.  Additional  details  of  the  items  listed  below  can  be  found  in 
the  pre-workshop  report.  Each  working  group  participant  should  familiarize  themselves  with 
the  pre-workshop  report  section  on  their  assigned  application  area.  It  is  expected  that  the  final 
workshop  report  will  address  the  issues  listed  below  for  each  application  area.  Of  course, 
working  groups  should  feel  free  to  address  issues  /  topics  which  are  not  contained  on  this 
checklist. 

Enabling  Technologies  -  building  block  technologies  required  for  application  development 

•  Augmented  or  hybrid  reality  displays 

•  Medical  imaging  technologies 

•  Non-medical  sensing  and  imaging  technologies 

•  Registration  methods 

•  Medical  image  processing  and  understanding 
Technical  Issues  -  common  to  each  of  the  application  areas 

•  System  validation  and  requirement  specification 

•  Minim  ally  invasive  procedure  design 

•  Design  for  safety 

Non-Technical  Issues  -  common  to  each  of  the  application  areas 

•  Clinical  evaluation  and  cost  benefit  analysis 

•  Acceptance 

•  Regulatory  issues 

•  Liability 

Pre- workshop  report  questions:  common  issues 

•  What  are  the  major  technical  problems  and  research  needs  in  RCAMI? 

•  What  areas  in  RCAMI  are  currently  NOT  receiving  enough  attention? 

•  What  piece  of  technology  do  you  wish  that  you  had  today? 

•  What  are  some  of  the  key  hurdles  to  overcome  in  sensing? 

•  What  are  some  of  the  key  hurdles  to  overcome  in  execution? 

•  What  are  some  of  the  key  hurdles  to  overcome  in  planning? 

•  What  will  be  possible  over  the  next  25  years? 
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Pre-workshop  report  questions:  application  areas 

•  What  applications  will  (your  application  area)  be  widely  use  for  in  the  next  decade? 
When  will  they  be  introduced? 

•  If  you  think  (your  application  area)  will  not  be  widely  used  over  the  next  decade,  why 
not? 

•  What  are  the  top  technical  problems  with  (your  application  area)l 

•  What  are  the  top  regulatory  problems  with  (your  application  area)l 

Questions  NOT  included  in  the  pre-workshop  report 

•  List  some  of  the  current  state  of  the  art  systems  in  (your  application  area). 

•  Where  can  collaboration  between  groups  be  established?  How  can  overlap  and  repeated 
effort  be  reduced? 

•  What  are  the  dependencies  and  overlaps  with  the  other  working  groups?  How  can  these  be 
resolved? 

•  Is  RCAMI  really  necessary?  Are  there  alternatives  which  are  simpler  or  less  costly? 

•  List  strengths  and/or  weaknesses  of  RCAMI. 

•  What  are  the  next  major  areas  of  research  towards  which  funding  should  be  directed? 

Terminology  Definitions 

•  Surgical  Simulators 

•  Virtual  reality 

•  Surgical  simulator 

•  Physical  modeling 

•  Pre-treatment  planning 

•  Image-Guided  Procedures 

•  Computer-assisted  surgery 

•  Computer-controlled  surgery 

•  Image-guided  surgery 

•  Robot-assisted  surgery 

•  Robotics  /  Manipulators 

•  Medical  robot 

•  Powered  and  unpowered  robotic  arm 

•  Passive,  semi-active  and  active  robotic  arm 

•  Passive  and/or  Active  compliance 

•  Interventional  robotic  arm 

•  Teleintervention 

•  Telepresence 

•  Telemedicine 

•  Telesurgery 

•  Teleintervention 

•  Telemanipulator 

•  Common  Issues 

•  Registration 
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•  Calibration 

•  Augmented  reality 

•  Hybrid  reality 

•  Minimally  invasive  procedure 
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Appendix  L  Workshop  Schedule 

ARRIVAL  -  SUNDAY,  JUNE  23,  1996 
Participant's  arrival/check  in. 

4:00  Welcome  Reception  -  Drawing  Room 

Speakers:  Anthony  DiGioia,  Takeo  Kanade  and 
Peter  Wells 

6:00  p.m.  Happy  Hour 

7:00  p.m.  Dinner 


DAY  ONE  -  MONDAY,  JUNE  24,  1996 


7:15  a.m. 

9:00  -9:30  a.m. 


9:30  -  10:00  a.m. 
10:00  -  10:30  a.m. 
10:30  -  10:45  a.m. 
10:45 -11:15  a.m. 
11:15-  11:45  a.m. 
11:45-  12:50  p.m. 


12:50  -2:00  p.m. 
2:00  -  3:30  p.m. 
3:30  -  3:50  p.m. 
3:50  -  5:30  p.m. 
6:15  p.m. 

7:15  p.m. 

8:30  p.m. 


Full  English  Breakfast 
Overview/Problem  Definition 

Speakers:  Anthony  M.  DiGioia  and  Takeo  Kanade 
Keynotes'  Address 

Edward  Chao,  Ph.D.  -  Surgical  Simulators 
Russ  Taylor,  Ph.D.  -  Robotics/Manipulators 
Break 

Philip  Green,  Ph.D.  -  Teleinterventions 

Stephane  Lavallee,  Ph.D.  -  Image  Guided  Procedures 

Special  Session  Speakers 

Safety  Issues  -  Brian  Davies 
Definitions  -  Patrick  Finlay 

Lunch 

Group  Breakout 
Break 

Group  Breakout 
Happy  Hour 
Dinner 

MRC-LINK  Keynote  Lecturer  -  Drawing  Room 
Speaker:  Dr.  Zigmund  Krukowski 


DAY  TWO  -  TUESDAY,  JUNE  25,  1996 


7:00  a.m. 

8:00  a.m.  -9:00 
9:00-11:30  a.m. 
11:30-  12:30  p.m. 
12:30  p.m. 


Full  English  Breakfast 
Group  Leader  Report 
Group  Breakout 
Group  Leader  Report 

Group  Photograph  -  Location  to  be  announced 
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12:45  -  1:30  p.m. 
1:30  -  7:00  p.m. 
7:00  p.m. 


Lunch 

Recreation/Free  Time 
Workshop  Finale 

Speakers:  Special  Trustees  for  United  Bristol  Hospitals  - 
Peter  N.T.  Wells 

National  Science  Foundation  -  Gil  Devey 
U.S.  Army  -  Major  Conrad  Clybum 


DAY  THREE  -  WEDNESDAY,  JUNE  26,  1996 


7:00  a.m. 

8:00  -  8:45  a.m. 


8:45  -  10:30  a.m. 
10:30  -  10:50  a.m. 
10:50-  11:50  a.m. 
11:50-  1:00  p.m. 
1:00  -  2:00  p.m. 


Full  English  Breakfast 

Special  Session  -  Technology  Transfer  Opportunities 

Speakers:  Faina  Shtem,  National  Cancer  Institute 
Group  Breakout 
Break 

Group  Leader  Final  Report 
Closing  Remarks 
Lunch 
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